uk.ac.starlink.util

## Class URLUtils

- java.lang.Object
- - uk.ac.starlink.util.URLUtils

-     public class URLUtils
      extends java.lang.Object

  Provides convenience methods for resolving URLs. This class provides some static methods for turning strings into URLs. This tends to be a bit of a pain in java, since you have to watch out for MalformedURLExceptions all over and work out what the context is. The methods provided here assume that if a string looks like a URL it is one, if it doesn't it's a file name, and if it's not absolute or resolved against a given context it is relative to the current directory. From the point of view of a user providing text to an application, or an XML document providing an href, this is nearly always what is wanted. The strategy can lead to surprising situations in the case that wacky URL protocols are used; for instance if `makeURL` is called on the string "gftp://host/file" and no gftp handler is installed, it will be interpreted as a file-protocol URL referring to the (presumably non-existent) file "gftp://host/file". In this case the eventual upshot will presumably be a file-not-found type error rather than a MalformedURLException type error getting presented to the user. Users of this class should be of the opinion that this is not a particularly bad thing.

  The `systemId` strings used by `Source`s have similar semantics to the strings which this class converts to URLs or contexts.

  This class assumes that the "file:" protocol is legal for URLs, and will throw AssertionErrors if this turns out not to be the case.

- - ### Method Summary

    All Methods Static Methods Concrete Methods
    Modifier and Type | Method and Description
    --- | ---
    `static java.net.URL` | `fixURL(java.net.URL url)` Fixes file: URLs which don't have enough slashes in them.
    `static java.net.URLConnection` | `followRedirects(java.net.URLConnection conn, int[] redirCodes)` Takes a URLConnection and repeatedly follows 3xx redirects until a non-redirect status is achieved.
    `static void` | `installCustomHandlers()` Attempts to install additional URL protocol handlers suitable for astronomy applications.
    `static java.net.URL` | `makeFileURL(java.io.File file)` Constructs a legal URL for a given File.
    `static java.net.URL` | `makeURL(java.lang.String location)` Obtains a URL from a string.
    `static java.net.URL` | `makeURL(java.lang.String context, java.lang.String location)` Obtains a URL from a string in a given context.
    `static java.net.URL` | `newURL(java.lang.String spec)` Drop-in replacement for the deprecated `URL.URL(String)` constructor.
    `static java.lang.String` | `percentEncodeIllegalCharacters(java.lang.String txt)` Percent-encodes those characters in a given string that are illegal anywhere in a URI.
    `static java.net.URL` | `resolveLocation(java.net.URL url0, java.lang.String location)` Returns the URL corresponding by a supplied location string, resolved in the context of a base URL.
    `static boolean` | `sameResource(java.net.URL url1, java.net.URL url2)` Attempts to determine whether two URLs refer to the same resource.
    `static java.lang.String` | `urlDecode(java.lang.String txt)` URL-decodes a string using UTF-8, without pesky exceptions.
    `static java.lang.String` | `urlEncode(java.lang.String txt)` URL-encodes a string using UTF-8, without pesky exceptions.
    `static boolean` | `urlEquals(java.net.URL url1, java.net.URL url2)` Compares two URLs.
    `static java.io.File` | `urlToFile(java.lang.String url)` Locates the local file, if any, represented by a URL.
    `static java.net.URI` | `urlToUri(java.net.URL url)` Turns a URL into a URI.

    - ### Methods inherited from class java.lang.Object

      `clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- - ### Method Detail

    - #### newURL

          public static java.net.URL newURL(java.lang.String spec)
                                     throws java.net.MalformedURLException

      Drop-in replacement for the deprecated `URL.URL(String)` constructor.

      All URL constructors are deprecated since Java 20 because of issues with parsing and validation. This utility method provides a way for code to avoid deprecation warnings. It may not do much to solve the underlying problems, and might introduce some new ones, but code that is having problems here can be adapted to handle URL creation more carefully; such approaches, according to the JDK documentation, should generally be URI-based. Other utility methods may be added here in future as required.

      As far as I can tell, most of the difficulties arising with URL parsing that have led to the deprecation relate to relatively strange URLs, so that "normal" http/https/file-protocol URL strings passed to this method should behave the same as if passed to the deprecated constructor. However, there may be changes of behaviour when it comes to constructions like embedded spaces in paths or special characters in query parts etc.

      Note that passing a string to this method which is not a valid URI, for instance because it contains unescaped illegal characters like "[", will fail, unlike the call to `new URL()`. In such cases a `MalformedURLException` will be thrown (which is really the result of a `URISyntaxException`).

      Parameters:
        `spec` - textual representation of URL
      Returns:
        URL, not null
      Throws:
        `java.net.MalformedURLException` - in case of syntax error

    - #### makeURL

          public static java.net.URL makeURL(java.lang.String location)

      Obtains a URL from a string. If the String has the form of a URL, it is turned directly into a URL. If it does not, it is treated as a filename, and turned into a file-protocol URL. In the latter case a relative or absolute filename may be used. If it is null or a blank string (or something else equally un-filename like?) then null is returned.
      Parameters:
        `location` - a string representing the location of a resource
      Returns:
        a URL representing the location of the resource

    - #### makeURL

          public static java.net.URL makeURL(java.lang.String context,
                                             java.lang.String location)

      Obtains a URL from a string in a given context. The string `context` is turned into a URL as per the `makeURL(String)` method, unless it is null or the empty string, in which case it is treated as a reference to the current directory. The string `location` is then turned into a URL in the same way as using `makeURL(String)`, except that if it represents a relative path it is resolved in the context of `context`, taking its protocol and/or relative position from it.
      Parameters:
        `context` - a string representing the context within which `location` is to be resolved
        `location` - a string representing the location of a resource
      Returns:
        a URL representing the location of the resource

    - #### urlToUri

          public static java.net.URI urlToUri(java.net.URL url)
                                       throws java.net.MalformedURLException

      Turns a URL into a URI.

      Since URIs are syntactically and semantically a superset of URLs, this conversion should not cause any errors. If, however, the input URL is malformed in rather extreme ways, then the URI construction will fail. These ways include (but are not necesssarily limited to) the features discussed in `URI.URI(String,String,String,String,String)`, namely that a scheme is present, but with a relative path, or that it has a registry-based authority part.

      Because of the way the class does the conversion, the method will itself resolve some malformations of URLs. You should not rely on this, however, firstly because the method might in principle change, but mostly because you should avoid creating such malformed URLs in the first place.

      The most common source of malformed URLs is that of `file` URLs which have inadequately escaped (windows) drive letters or spaces in the name: such URLs should be constructed using the `File.toURI()` or `File.toURL()` methods. Such URLs will be escaped by this method.

      Parameters:
        `url` - a URL to be converted. If this is null, then the method returns null
      Returns:
        the input URL as a URI, or null if the input was null
      Throws:
        `java.net.MalformedURLException` - if the URI cannot be constructed because the input URL turns out to be malformed

    - #### makeFileURL

          public static java.net.URL makeFileURL(java.io.File file)

      Constructs a legal URL for a given File. Unlike java, this gives you a URL which conforms to RFC1738 and looks like "`file://localhost/abs-path`" rather than "`file:abs-or-rel-path`".
      Parameters:
        `file` - file
      Returns:
        URL
      See Also:
        "RFC 1738"

    - #### fixURL

          public static java.net.URL fixURL(java.net.URL url)

      Fixes file: URLs which don't have enough slashes in them. Java generates invalid URLs of the form "`file:abs-or-rel-path`" when it should generate "`file://localhost/abs-path`".
      Parameters:
        `url` - input URL
      Returns:
        fixed URL
      See Also:
        "RFC 1738"

    - #### sameResource

          public static boolean sameResource(java.net.URL url1,
                                             java.net.URL url2)

      Attempts to determine whether two URLs refer to the same resource. Not likely to be foolproof, but slightly smarter than using `equals`.
      Parameters:
        `url1` - first URL
        `url2` - second URL
      Returns:
        true if `url1` and `url2` appear to refer to the same resource

    - #### urlToFile

          public static java.io.File urlToFile(java.lang.String url)

      Locates the local file, if any, represented by a URL. If the URL string uses the "file:" protocol, and has no query or anchor parts, the filename will be extracted and the corresponding file returned. Otherwise, null is returned.
      Parameters:
        `url` - URL string
      Returns:
        local file referenced by `url`, or null

    - #### urlEquals

          public static boolean urlEquals(java.net.URL url1,
                                          java.net.URL url2)

      Compares two URLs. This does approximatly the same job as the URL.equals() method, but it avoids the possible network accesses associated with that implementation, and copes with null values.
      Parameters:
        `url1` - first URL
        `url2` - second URL
      Returns:
        true iff both are the same, or both are null

    - #### urlEncode

          public static java.lang.String urlEncode(java.lang.String txt)

      URL-encodes a string using UTF-8, without pesky exceptions.
      Parameters:
        `txt` - string to encode
      Returns:
        encoded string
      See Also:
        `URLEncoder`

    - #### urlDecode

          public static java.lang.String urlDecode(java.lang.String txt)

      URL-decodes a string using UTF-8, without pesky exceptions.
      Parameters:
        `txt` - string to decode
      Returns:
        decoded string
      See Also:
        `URLDecoder`

    - #### percentEncodeIllegalCharacters

          public static java.lang.String percentEncodeIllegalCharacters(java.lang.String txt)

      Percent-encodes those characters in a given string that are illegal anywhere in a URI. Character coding is UTF-8.
      Parameters:
        `txt` - raw string
      Returns:
        string with illegal characters percent-encoded
      See Also:
        RFC 3986, section 2

    - #### followRedirects

          public static java.net.URLConnection followRedirects(java.net.URLConnection conn,
                                                               int[] redirCodes)
                                                        throws java.io.IOException

      Takes a URLConnection and repeatedly follows 3xx redirects until a non-redirect status is achieved. Infinite loops are defended against. The Accept-Encoding header, if present, is propagated to redirect targets.

      Note that the `HttpURLConnection.setInstanceFollowRedirects(boolean)` method does something like this, but it refuses to redirect between different URL protocols, for security reasons (see http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4620571). Considering similar arguments, this method will direct HTTP->HTTPS, but not vice versa.

      Parameters:
        `conn` - initial URL connection
        `redirCodes` - list of HTTP codes for which redirects should be followed; if null all suitable 3xx redirections will be followed (301, 302, 303, 307, 308)
      Returns:
        target URL connection (if no redirects, the same as `hconn`)
      Throws:
        `java.io.IOException`
      See Also:
        RFC 9110 Sec 15.4

    - #### resolveLocation

          public static java.net.URL resolveLocation(java.net.URL url0,
                                                     java.lang.String location)
                                              throws java.io.IOException

      Returns the URL corresponding by a supplied location string, resolved in the context of a base URL.
      Parameters:
        `url0` - context URL
        `location` - location, may be relative or absolute URI
      Returns:
        resolved location URL
      Throws:
        `java.io.IOException` - if anything goes wrong

    - #### installCustomHandlers

          public static void installCustomHandlers()

      Attempts to install additional URL protocol handlers suitable for astronomy applications. Currently installs handlers which can supply MySpace connections using either "`ivo:`" or "`myspace:`" protocols.