Our approach is based on the methodology of Nguyen et al. [NLF18] who perform an empirical study of web browser caches. The first goal of this methodology was to define a comprehensive test structure, which allows to derive a considerable test suite for analyzing the reliability of web caching systems. To do so, Nguyen et al. conducted an in-depth review of RFC 7234 as well as publications related to HTTP and web caching. As a result they proposed a test suite structure with the following topics:
1. Explicit caching controlled by server 2. Implicit caching controlled by cache 3. Client-originated caching policies 4. Cache key adaptation
5. Invalidation of freshness 6. Partial content
7. Security 8. Other
On the basis of this methodology and a proposed test case specification language Nguyen et al. were able to identify 397 test cases covering constructive as well as destructive tests. Even though their paper covers web browser caches only, the test collection can also be utilized for analyzing all kinds of web caching systems. In this paper, we utilize their methodology, test suite and test case specification language to build a web cache testing tool for investigating proxy caches and CDNs.
5.5.1 Architecture
Figure5.2shows the architecture of our web cache testing system which is based on the approach of [NLF18]. We implement our tool as functional black box test with the main test objective being the cache. The architecture of the tool consists of a test server, which is accessed and controlled by the test client. Both endpoints need to be under control of the test environment.
The proposed architecture enables to explore forward or reverse proxies and CDNs. The server-internal caches WP Super Cache and Ehcache cannot be tested by our system, though, as these web caching systems are different to others. Here, the caching systems are not designed to store responses according to protocol header elements of HTTP messages. Instead, the caching behavior is set via internal elements of the corresponding programming environment.
Result ...
Cache Cache Testing Server
Cache Testing Client
Frontend Backend
Request Processor
Test cases &
Target host Request 1 Request 1
Response 1 Response 1
Request 2 Request 2
Response 2 Response 2
Result 1
Result 2
Evaluation
Request ... Request ...
Response ...
Response ...
Figure 5.2: Architecture of universal shared cache testing system based on [NLF18]
The cache testing client is a web application which consists of a frontend where the test vectors can be defined and a backend which executes the requests as well as evaluates the returning responses. The frontend provides a user interface which requires two mandatory input arguments:
the target host and a set of test cases defined by the provided test case specification language.
The request processor reads the supplied test cases, converts each test case into a series of requests and schedules them for delivery to the server. On their way to the server, the requests pass the cache. In order to unambiguously distinguish whether a response is originated by the cache testing server or supplied by the web caching system, the request processor includes specific meta information to the requests which will be replayed by the cache testing server.
By means of the replayed meta information included in the responses, our cache testing client is able to infer several properties including whether the content is a cached copy or if other expected testing requirements are met.
5.5.2 Test Case Suite
The developed testing tool for shared web caches comes with a test case suite of 397 tests that can be used out of the box. This suite originates from [NLF18] and tests the conformance with various HTTP and web caching standards. All 397 tests are defined using a custom specification language. By this means, the base test suite can be easily explored and even extended with individual tests. Listing5.1shows an example test case which is defined with the proposed test case specification language.
1 ## Testing freshness lifetime of 60 seconds
2 GET /rsc -c ’Accept:application/json;Cache-Control:no-store’
3 -s ’Cache-Control:max-age=60’
4 -p 30
5 -e ’ch:false;st:200’
6 GET /rsc -c ’Accept:application/json’
7 -e ’ch:true;st:200’
8 -p 35
9 GET /rsc -c ’Accept:application/json’
10 -e ’ch:false;st:200’
Listing 5.1: Example test case defined by the provided specification language
The provided test case specification language is based on the command line tool curl [Ste18].
Each test case starts by a descriptive title followed by one or multiple commands (see Listing5.1).
A command contains two mandatory and several optional arguments. The first mandatory argument is the request method, i.e. GET, POST, PUT, PATCH and DELETE. The second mandatory argument represents the request target which is the URL omitting the scheme and host parts. Listing5.1contains three commands: lines 2 to 5 define the first command and the second one is specified from lines 6 to 8. Lines 9 to 10 cover the third and last command. Each command specifies a request and its expected response. The first command demands to generate aGETrequest targeting the URL/rsc. Optionally, the test case specification language allows to add one or more headers to the request. This is done by the-cparameter flag. Multiple headers must be separated by semicolon. In the example, the first command requires to add two headers to the request: (i) theAcceptheader with the valueapplication/jsonand (ii) the Cache-Control header containing the valueno-store. The -sparameter flag instructs the cache testing server to add the specified headers to the resulting response. The-p parameter flag forces the client to wait a certain amount of seconds before executing the next request. Such breaks between requests are, e.g., required for analyzing whether a cache complies with the freshness lifetime requirements. With the-eparameter flag, users can define the to be expected properties of the response. These can be either RFC 7234 requirements or expectations of the own caching policy. The first expected property in the first commandch:false(see line 5) assumes that the resulting response originates from the server and is not a cached copy.
The second expected propertyst:200demands the response to contain the status code200 Ok.
Figure 5.3 shows the request/response flow executed by the web cache testing tool when executing the test case specified in Listing 5.1. Accordingly, the first request includes the GETmethod targeting the URL path/rsc. Note that theGETrequest contains theAccept andCache-Controlheaders signalling the retrieval of anapplication/jsonresource representation that must not be a stored copy from a cache. The request also includes the X-Responseheader which contains the headers that the cache testing server has to add to the resulting response. Therefore, the resulting response includes theCache-Controlheader with the valuemax-age=60which allows all caches to store the content for 1 minute.
All requests created by a our cache testing client comprise the X-Id header with a unique random generated id. This token is returned by the cache testing server in case the request reaches it. The cache testing server adds the id in theX-Idheader of the response as well as the body. The generated id including in the request and its resulting response allows evaluating whether a received response has been replayed by a caching system or originates from our cache testing server. For instance, by means of the id inside theX-Id header, our tool can assess that the first response is sourced from the server as it contains the same id as the corresponding
Cache Testing
Client
GET/rsc HTTP/1.1 Host: example.org Accept: application/json Cache-Control: no-store
X-Response: Cache-Control: max-age=60 X-Id: 142661fd2252eea7
GET/rscHTTP/1.1 Host: example.org Accept: application/json Cache-Control: no-store
X-Response: Cache-Control: max-age=60 X-Id: 142661fd2252eea7
HTTP/1.1 200 OK
Cache-Control: max-age=60 Content-Length: 26
Date: Wed, 29 Aug 2018 13:29:28 GMT X-Id: 142661fd2252eea7
{“Id“: “142661fd2252eea7“}
HTTP/1.1 200 OK
Cache-Control: max-age=60 Content-Length: 26
Date: Wed, 29 Aug 2018 13:29:28 GMT X-Id: 142661fd2252eea7
{“Id“: “142661fd2252eea7“}
GET/rsc HTTP/1.1 Host: example.org Accept: application/json X-Id: 1aed7eb71b1fe040
HTTP/1.1 200 OK
Cache-Control: max-age=60 Content-Length: 26
Date: Wed, 29 Aug 2018 13:29:28 GMT X-Id: 142661fd2252eea7
Age: 30
{“Id“: “142661fd2252eea7“}
GET/rsc HTTP/1.1 Host: example.org Accept: application/json X-Id: 635ad9e0cafb11d7
GET/rsc HTTP/1.1 Host: example.org Accept: application/json X-Id: 635ad9e0cafb11d7
HTTP/1.1 200 OK Content-Length: 26
Date: Wed, 29 Aug 2018 13:30:33 GMT X-Id: 635ad9e0cafb11d7
{“Id“: “635ad9e0cafb11d7“}
HTTP/1.1 200 OK Content-Length: 26
Date: Wed, 29 Aug 2018 13:30:33 GMT X-Id: 635ad9e0cafb11d7
{“Id“: “635ad9e0cafb11d7“}
30s later
35s later
✓
✓
✓
Cache
Cache Testing
Server
Figure 5.3: Request/Response flow produced by the web cache testing tool according to the test case specified in Listing5.1
request. The second response instead is a recycled copy from a shared cache (emphasized by the gray background), since it includes the id of the first request and not the one from the second request. Another indicator for a cached response is theAgeheader. Moreover, the cache testing server also appends theDateheader which represents the response creation time. This date value is also another information, which can be used to assess whether the response is reused or new.
Figure5.3illustrates that all the three responses comply with the expected properties specified in the test case specification (see Listing5.1). The first response is a new one from the server and contains the status code200 Ok. The second is a recycled response from the web caching systems comprising the status code200 Okas well. The final response is a new one forwarded by the cache from the server as the previous reused response is expired.
The introduced web cache testing tool with the base test suite is provided as public domain software can be obtained fromhttps://github.com/DASCologne/Cache-Testing-Tool.