Java HTTP Client - Configuration
Here is an index to configuration topics, some of which are on this page, others having their own page:
This gives instructions for setting the parameters for the Java HTTP client.
All of the methods shown are on
Many of the configuration parameters below can also be set using the system properties, see the Javadoc for HttpURLConnection for those details.
Note: All of the timeout values are specified in milliseconds. 1000 milliseconds equals one second.
The connection timeout mechanism allows you to limit the amount of time you wait for the connection to be established with the server. When you make an HTTP request, the HTTP client may immediately attempt to create a TCP connection with the server, or if the maximum number of connections to the given server has been reached, the HTTP client will wait for an existing connection to become available.
This timeout applies both to the time it takes an underlying socket connection to be established, and to the time that you have to wait to get a currently established connection.
If you are using JDE 1.4 or higher, this timeout is the value set
as the timeout value when setting up the socket
The request timeout is the amount of time from when the request is initiated until the response has been read. This includes all of the time to establish the connection.
Most TCP connections that support HTTP requests service more than one request in sequence and are therefore kept alive after a request is finished. Most HTTP servers kill these idle connections after a specific interval of inactivity. The HTTP client can also terminate the idle connection if no request has been sent on it for a specified period of time.
When an idle connection is terminated by the server, the client does not get any notification of this fact, and it is detected the next time an HTTP request is sent on the connection. This is undesirable, because that HTTP request will either fail or have to be retried.
Because of this, it is generally desirable to kill idle connections in the HTTP client just before they are killed by the server. The default for the idle connection timeout is set to 14 seconds, which is one second less than the default of the idle connection timeout of the Apache web server.
There are times when you cannot be sure of the idle connection timeout value in the server, and may therefore have to risk sending an HTTP request on a connection that might have been killed by the server. In the case of an HTTP GET, this is not a problem, because if it is sent on a dead connection it can automatically be retried.
However, an HTTP POST can never be retried because the semantics of the HTTP POST require it to be sent only once. To ensure the best chance of success for a POST, we can send out a "ping" message on the connection just before the POST request, which will either detect a dead connection, or keep the connection alive long enough to send the POST successfully.
If you can't reliably use the idle connection timeout parameter (because you don't know the server environment), you should set this value to a relatively small value like 1000 milliseconds (one second).
This is the number of times an idempotent HTTP request is attempted to be sent before giving up. An idempotent HTTP request is any HTTP request except for POST. This parameter is also the number of attempts to send the idle connection ping before a POST request.
The default value for this is 3, which is one greater than the default maximum number of connections for a host/port. If you want to guarantee that a request will be sent on a working connection, set this value to one greater than the maximum number of connections per host/port. Doing this insures that even if all of the existing connections between the host/port are dead (and they are all tried), one new more connection will be created to allow the request to be sent.
This is the maximum number of connections to be kept alive for any host/port combination. It is a global parameter which applies to all host/port combinations. The default is 2, as this is what is recommended for use on the Internet in RFC 2616 section 8.1.4. However, if you are using the HTTP client in certain high throughput applications, a much higher value might be appropriate. In setting this value, be sure and consider the Tries value.
In addition to respecting all of the Java APIs and properties (see the
HttpURLConnection documentation) related to setting
the proxy, the
Basic, Digest and NTLM authentication are supported. In order to use
these authentication modes, implement an instance of the
If you already know that you will require authentication, it's better to
When when using the output streaming or pipelining features, setting the authentication type beforehand is essential, as this allows streaming or pipelining to progress only after authentication has completed. This is particularly important for NTLM authentication which is connection based. Once the underlying socket connection is authenticated with NTLM, any HTTP request can be executed on it (using the same credential) and it requires no further authentication.
By default, when using authentication for connections, each host/port (or proxy host/port) is associated with a single set of credentials that is requested using the HttpUserAgent only once. After being initially requested (and assuming the credentials are valid), any subsequent HttpURLConnections on that host/port (or proxy host/port) will use the same credentials. Doing this allows the HttpUserAgent to be directly hooked to a user interface that asks the user to specify the credentials for a given host and port.
There are some situations where you may wish to have different sets of
authentication credentials for connections going to the same host/port.
You can do this by calling
The JRE supports SOCKS socket connections. To use a SOCKS connection
with the HTTP client, you must create a custom socket factory which creates
the correct type of
Make a subclass of
You can set the SOCKS configuration either using the
By default, any data sent with the request is buffered prior to being
sent, which results in a double copy of the data. This can be avoided
by the use of the
If you are using authentication, you should also set the
authentication type (Basic, Digest, or NTLM).
this is used with streaming, it will complete the authentication
process before streaming the request. If you don't set the
authentication type in advance, the request will fail with an
In some applications, it might be critical to get a response right away,
and there are many different servers available to handle the request.
In this case, the use of the connection timeout,
and request timeout are recommended. Set these
to relatively small values, and in the event of a timeout, you create
a new HTTP connection to a different server. Alternatively, you can
use a different proxy, and set the proxy using the
Most proxy servers support the FTP protocol for the purpose of accessing files or directories. You can use this simply by specifying "ftp:" as the protocol instead of "http:" in your URL, and of course specify the proxy server using the usual means.
However, you must use the com.oaklandsw.http.HttpURLConnection.openConnection() method to open the connection. If you don't, the URL provided is selected based on the protocol and you will not get the Oakland Software implementation.
When using a proxy server, you can use the CONNECT method to use the underlying TCP stream for an arbitrary purpose through the proxy. This is called HTTP Tunneling.
When using the CONNECT method, you can stream using both the HttpURLConnection's
input and output streams by calling
Here is some sample code to illustrate:
The Java HTTP client supports all of the WebDAV methods defined by the following RFCs:
The Java HTTP client allows any method to be defined in addition to the built-in
HTTP and WebDAV methods. This is accomplished by setting the properties
associated with a method using the
These versions of the JRE are supported by the HTTP client version 1.9.0. Version 1.9.0 does not include the following features:
If you require this version, please it.