Module org.htmlunit
Package org.htmlunit

Class WebRequest

java.lang.Object
org.htmlunit.WebRequest
All Implemented Interfaces:
Serializable
public class WebRequest extends Object implements Serializable
Parameter object for making web requests.
Author:
Brad Clarke, Hans Donner, Ahmed Ashour, Marc Guillemot, Rodney Gitzel, Ronald Brill, Adam Afeltowicz, Joerg Werner, Michael Lueck, Lai Quang Duong, Kristof Neirynck
See Also:

  • Constructor Details

    • WebRequest

      public WebRequest(URL url, String acceptHeader, String acceptEncodingHeader)
      Creates or updates this object.. Instantiates a WebRequest for the specified URL.
      Parameters:
      url - the target URL
      acceptHeader - the accept header to use
      acceptEncodingHeader - the accept encoding header to use

    • WebRequest

      public WebRequest(URL url, Charset charset, URL refererUrl)
      Creates or updates this object.. Instantiates a WebRequest for the specified URL.
      Parameters:
      url - the target URL
      charset - the charset to use
      refererUrl - the url be used by the referer header

    • WebRequest

      public WebRequest(URL url)
      Creates or updates this object.. Instantiates a WebRequest for the specified URL.
      Parameters:
      url - the target URL

    • WebRequest

      public WebRequest(URL url, HttpMethod submitMethod)
      Creates or updates this object.. Instantiates a WebRequest for the specified URL using the specified HTTP submit method.
      Parameters:
      url - the target URL
      submitMethod - the HTTP submit method to use

  • Method Details

    • newAboutBlankRequest

      public static WebRequest newAboutBlankRequest()
      Returns a new request for about:blank.
      Returns:
      a new request for about:blank

    • getUrl

      public URL getUrl()
      Returns the target URL. Returns the target URL.
      Returns:
      the target URL

    • setUrl

      public void setUrl(URL url)
      Creates or updates this object. Sets the target URL. The URL may be simplified if needed (for instance eliminating irrelevant path portions like "/./").
      Parameters:
      url - the target URL

    • getProxyHost

      public String getProxyHost()
      Returns the proxy host to use. Returns the proxy host to use.
      Returns:
      the proxy host to use

    • setProxyHost

      public void setProxyHost(String proxyHost)
      Creates or updates this object. Sets the proxy host to use.
      Parameters:
      proxyHost - the proxy host to use

    • getProxyPort

      public int getProxyPort()
      Returns the proxy port to use. Returns the proxy port to use.
      Returns:
      the proxy port to use

    • setProxyPort

      public void setProxyPort(int proxyPort)
      Creates or updates this object. Sets the proxy port to use.
      Parameters:
      proxyPort - the proxy port to use

    • getProxyScheme

      public String getProxyScheme()
      Returns the proxy scheme to use. Returns the proxy scheme to use.
      Returns:
      the proxy scheme to use

    • setProxyScheme

      public void setProxyScheme(String proxyScheme)
      Creates or updates this object.. Sets the proxy scheme to use.
      Parameters:
      proxyScheme - the proxy scheme to use

    • isSocksProxy

      public boolean isSocksProxy()
      Returns whether SOCKS proxy or not. Returns whether SOCKS proxy or not.
      Returns:
      whether SOCKS proxy or not

    • setSocksProxy

      public void setSocksProxy(boolean isSocksProxy)
      Creates or updates this object.. Sets whether SOCKS proxy or not.
      Parameters:
      isSocksProxy - whether SOCKS proxy or not

    • getTimeout

      public int getTimeout()
      Returns the timeout to use.
      Returns:
      the timeout to use

    • setTimeout

      public void setTimeout(int timeout)
      Creates or updates this object.. Sets the timeout to use.
      Parameters:
      timeout - the timeout to use

    • getEncodingType

      public FormEncodingType getEncodingType()
      Returns the form encoding type to use. Returns the form encoding type to use.
      Returns:
      the form encoding type to use

    • setEncodingType

      public void setEncodingType(FormEncodingType encodingType)
      Creates or updates this object.. Sets the form encoding type to use.
      Parameters:
      encodingType - the form encoding type to use

    • getParameters

      public List<NameValuePair> getParameters()
      Returns the request parameters to use.

      Retrieves the request parameters used. Similar to the servlet api function getParameterMap() this works depending on the request type and collects the url parameters and the body stuff.
      The value is also normalized - null is converted to an empty string.

      In contrast to the servlet api this creates a separate KeyValuePair for every parameter. This means that pairs with the same name can be part of the list. The servlet api will return a string[] as value for the key in this case.
      Additionally this method includes also the uploaded files for multipart post requests.

      Returns:
      the request parameters to use

    • getRequestParameters

      public List<NameValuePair> getRequestParameters()
      Returns the request parameters to use. INTERNAL API - SUBJECT TO CHANGE AT ANY TIME - USE AT YOUR OWN RISK.
      Retrieves the request parameters to use. If set, these request parameters will overwrite any request parameters which may be present in the URL. Should not be used in combination with the request body.
      Returns:
      the request parameters to use

    • setRequestParameters

      public void setRequestParameters(List<NameValuePair> requestParameters) throws RuntimeException
      Creates or updates this object.. INTERNAL API - SUBJECT TO CHANGE AT ANY TIME - USE AT YOUR OWN RISK.
      Sets the request parameters to use. If set, these request parameters will overwrite any request parameters which may be present in the URL. Should not be used in combination with the request body.
      Parameters:
      requestParameters - the request parameters to use
      Throws:
      RuntimeException - if the request body has already been set

    • getRequestBody

      public String getRequestBody()
      Returns the body content to be submitted if this is a POST request. Returns the body content to be submitted if this is a POST request. Ignored for all other request types. Should not be used in combination with request parameters.
      Returns:
      the body content to be submitted if this is a POST request

    • setRequestBody

      public void setRequestBody(String requestBody) throws RuntimeException
      Creates or updates this object.. Sets the body content to be submitted if this is a POST, PUT or PATCH request. Other request types result in RuntimeException. Should not be used in combination with request parameters.
      Parameters:
      requestBody - the body content to be submitted if this is a POST, PUT or PATCH request
      Throws:
      RuntimeException - if the request parameters have already been set or this is not a POST, PUT or PATCH request.

    • getHttpMethod

      public HttpMethod getHttpMethod()
      Returns the HTTP submit method to use. Returns the HTTP submit method to use.
      Returns:
      the HTTP submit method to use

    • setHttpMethod

      public void setHttpMethod(HttpMethod submitMethod)
      Creates or updates this object.. Sets the HTTP submit method to use.
      Parameters:
      submitMethod - the HTTP submit method to use

    • getAdditionalHeaders

      public Map<String,String> getAdditionalHeaders()
      Returns the additional HTTP headers to use. Returns the additional HTTP headers to use.
      Returns:
      the additional HTTP headers to use

    • setAdditionalHeaders

      public void setAdditionalHeaders(Map<String,String> additionalHeaders)
      Creates or updates this object.. Sets the additional HTTP headers to use.
      Parameters:
      additionalHeaders - the additional HTTP headers to use

    • isAdditionalHeader

      public boolean isAdditionalHeader(String name)
      Creates or updates this object.. Returns whether the specified header name is already included in the additional HTTP headers.
      Parameters:
      name - the name of the additional HTTP header
      Returns:
      true if the specified header name is included in the additional HTTP headers

    • getAdditionalHeader

      public String getAdditionalHeader(String name)
      Creates or updates this object.. Returns the header value associated with this name.
      Parameters:
      name - the name of the additional HTTP header
      Returns:
      the value or null

    • setRefererHeader

      public void setRefererHeader(URL url)
      Creates or updates this object.. Sets the referer HTTP header - only if the provided url is valid.
      Parameters:
      url - the url for the referer HTTP header

    • getFetchDestination

      public WebRequest.FetchDestination getFetchDestination()
      Returns the destination of this request, used to compute the Sec-Fetch-Dest header (and, unless overridden, the default Sec-Fetch-Mode). Defaults to WebRequest.FetchDestination.EMPTY, which is correct for plain XMLHttpRequest/fetch() calls.
      Returns:
      the destination of this request

    • setFetchDestination

      public void setFetchDestination(WebRequest.FetchDestination fetchDestination)
      Sets the destination of this request.
      Parameters:
      fetchDestination - the destination of this request, or null to reset to WebRequest.FetchDestination.EMPTY

    • getFetchModeOverride

      public WebRequest.FetchMode getFetchModeOverride()
      Returns the explicit mode override for this request, if any. When null (the default), the mode is derived from the destination.
      Returns:
      the mode override, or null if none was set

    • setFetchModeOverride

      public void setFetchModeOverride(WebRequest.FetchMode fetchMode)
      Sets an explicit mode override for this request, e.g. for a fetch() call using an explicit mode option, or a subresource request using the crossorigin attribute (which forces CORS mode).
      Parameters:
      fetchMode - the mode to use, or null to derive it from the destination

    • isUserActivation

      public boolean isUserActivation()
      Returns whether this request is the result of a navigation backed by genuine user activation (e.g. a click on a link, a typed URL, or a form submitted via a click on its submit button) as opposed to one triggered purely by script (e.g. location.href = ..., a <meta http-equiv="refresh">, or an automatically-loaded <iframe>).

      Only relevant for requests whose Sec-Fetch-Mode is navigate; used to compute the presence of the Sec-Fetch-User header, which real browsers omit entirely (never send as ?0) whenever this is false.

      Returns:
      whether this request was triggered by a real user gesture

    • setUserActivation

      public void setUserActivation(boolean userActivation)
      Sets whether this request is the result of a navigation backed by genuine user activation.
      Parameters:
      userActivation - whether this request was triggered by a real user gesture

    • getRequestingUrl

      public URL getRequestingUrl()
      Returns the URL of the document or script that initiated this request, used to compute the Sec-Fetch-Site header. null means there is no initiator (e.g. a browser-chrome-initiated navigation such as a typed URL or bookmark), which maps to Sec-Fetch-Site: none.

      Note this is tracked separately from the Referer header: unlike the referrer, it must not be affected by referrer-policy stripping, since Sec-Fetch-Site always reflects the true relationship between the initiator and the target, even when no Referer header is sent.

      Returns:
      the URL of the initiator, or null if there is none

    • setRequestingUrl

      public void setRequestingUrl(URL requestingUrl)
      Sets the URL of the document or script that initiated this request.
      Parameters:
      requestingUrl - the URL of the initiator, or null if there is none

    • markAsNavigation

      public void markAsNavigation(URL requestingUrl, boolean userActivation)
      Convenience method for the common case of a top-level navigation (an anchor/area click, a form submission, a script-driven location change, ...): sets WebRequest.FetchDestination.DOCUMENT, the initiator URL, and whether the navigation was backed by genuine user activation, all in one call.

      Every navigation-triggering call site needs all three of these set together for correct Sec-Fetch-* headers; bundling them here makes it harder for a call site to set some of them and forget the rest.

      For navigations whose destination is not WebRequest.FetchDestination.DOCUMENT (e.g. an <iframe>/<frame> load), use markAsNavigation(FetchDestination, URL, boolean) instead.

      Parameters:
      requestingUrl - the URL of the page initiating this navigation, or null if there is none (e.g. a typed URL)
      userActivation - whether this navigation was triggered by a real user gesture as opposed to script

    • markAsNavigation

      public void markAsNavigation(WebRequest.FetchDestination destination, URL requestingUrl, boolean userActivation)
      Same as markAsNavigation(URL, boolean), but for navigations whose destination isn't a top-level WebRequest.FetchDestination.DOCUMENT - currently only <iframe>/<frame> loads (WebRequest.FetchDestination.IFRAME/ WebRequest.FetchDestination.FRAME).
      Parameters:
      destination - the navigation's destination
      requestingUrl - the URL of the page initiating this navigation, or null if there is none
      userActivation - whether this navigation was triggered by a real user gesture as opposed to script

    • setAdditionalHeader

      public void setAdditionalHeader(String name, String value)
      Creates or updates this object.. Sets the specified name/value pair in the additional HTTP headers.
      Parameters:
      name - the name of the additional HTTP header
      value - the value of the additional HTTP header

    • removeAdditionalHeader

      public void removeAdditionalHeader(String name)
      Creates or updates this object.. Removed the specified name/value pair from the additional HTTP headers.
      Parameters:
      name - the name of the additional HTTP header

    • getUrlCredentials

      public org.apache.http.auth.Credentials getUrlCredentials()
      Returns the credentials if set as part of the url. Returns the credentials to use.
      Returns:
      the credentials if set as part of the url

    • getCredentials

      public org.apache.http.auth.Credentials getCredentials()
      Returns the credentials if set from the external builder. Returns the credentials to use.
      Returns:
      the credentials if set from the external builder

    • setCredentials

      public void setCredentials(org.apache.http.auth.Credentials credentials)
      Creates or updates this object.. Sets the credentials to use.
      Parameters:
      credentials - the credentials to use

    • getCharset

      public Charset getCharset()
      Returns the character set to use to perform the request. Returns the character set to use to perform the request.
      Returns:
      the character set to use to perform the request

    • setCharset

      public void setCharset(Charset charset)
      Creates or updates this object. Sets the character set to use to perform the request. The default value is StandardCharsets.ISO_8859_1.
      Parameters:
      charset - the character set to use to perform the request

    • getDefaultResponseContentCharset

      public Charset getDefaultResponseContentCharset()
      Returns the default character set to use for the response when it does not specify one.
      Returns:
      the default character set to use for the response when it does not specify one.

    • setDefaultResponseContentCharset

      public void setDefaultResponseContentCharset(Charset defaultResponseContentCharset)
      Creates or updates this object. Sets the default character set to use for the response when it does not specify one.

      Unless set, the default is StandardCharsets.UTF_8.

      Parameters:
      defaultResponseContentCharset - the default character set of the response

    • hasHint

      public boolean hasHint(WebRequest.HttpHint hint)
      Creates or updates this object..
      Parameters:
      hint - the hint to check for
      Returns:
      true if the hint is enabled

    • addHint

      public void addHint(WebRequest.HttpHint hint)
      Creates or updates this object.. Enables the hint.
      Parameters:
      hint - the hint to add

    • toString

      public String toString()
      Returns a string representation of this object. Returns a string representation of this object.
      Overrides:
      toString in class Object
      Returns:
      a string representation of this object