This work tackles the problem of test case generation from scenarios using Model-Driven Testing approach. From the design point of view, we see a RESTful application as consisting of two aspects: a structural aspect, which deals with the data structure of the resources ex-posed by the application, as well as CRUD operations over these resources (i.e. a resource model), and a dynamic part, which deals with determining which operations can be applied to a resource given its current state (i.e. a resource lifecycle model). The former aspect is usually captured by means of annotated class diagrams while the latter can be captured by means of state chart diagrams. We contend that existing tools (e.g. Apiary blueprint-related, swagger-related, etc.) cover only the structural aspect.
The thesis proposes a tool to automate the generation of test cases that exercise the applica-tion by providing class diagrams and state charts in the form of textual Domain Specific Languages (DSLs). More concretely, we design domain specific languages embarked as a tool that produces test code for Java applications using the Spring MVC framework. We consider Gherkin language as the DSL used as it could capture the structural as well as behavioural aspects of the application. Moreover, the Gherkin language can be written in plain spoken language that can be understood by end users. Using this tool, the developers have to provide the resource and its lifecycle models in the form of a Gherkin feature file and it will generate the test cases automatically. Additionally, we provide the generation of a mock controller which are tested by default with the generated test cases. We contend that the generated test cases using our approach alleviate not only the effort in writing the test code but also in bringing the benefits of TDD, by checking that all the RESTful interactions are properly implemented. Unlike the existing tools that cover only the structural aspects of a hypermedia REST application, this solution we propose covers the application’s structural as well as dynamic aspects.
6 1.3 Objectives
From the above, we identify the following general objective:
Developing a domain specific language that allows programmers to specify the in-teraction protocol of a RESTful application and the set of tools to generate testing artefacts from the interaction protocols thus specified.
The above can be further refined in the following specific objectives:
- Designing a domain specific language for specifying RESTful interactions - Implementing code generators for:
o Test case suites for exercising the protocol implementing the RESTful inter-actions
o A mock testing implementation that mimics the actual RESTful application according to the examples specified using the domain specific language As a proof concept, we will consider code generation for RESTful applications written with Spring Boot framework. The latter implies that we target applications written in Java and more specifically, enterprise applications built on top of Spring MVC framework.
1.4 Document Outline
The rest of this thesis report is structured like the following.
Chapter 2 introduces the various theories and concepts which will be used to discuss the approach throughout the thesis report.
Chapter 3 summarizes the various researches related to the context under discussion and explains the existing tools providing similar functionalities as our aimed tool.
Chapter 4 provides an overview of the architecture used in the approach and provides an initial idea about the planned method of implementation of the tool.
Chapter 5 is probably the most important chapter of the thesis as a tool will be developed step by step using the approach discussed.
In chapter 6, the implemented tool will be evaluated in order to verify how much the tool can be used to solve real-world problems. It also provides the existing limitations of the tool.
Chapter 7 summarizes the thesis as a whole, providing the final outcomes from the imple-mentation and validation perspective. It also provides some suggestions for future work.
7
2 Background
In this chapter, some of the important theory and concepts, based on which the thesis dis-cussion will be progressed, are briefly described. First, we discuss the main concept of this thesis, which is Representational State Transfer or REST and its principles. Then we duce the concept of Test-Driven Development. The discussion is followed by a brief intro-duction to class diagrams and statechart diagrams. Then we introduce the concept of Domain Specific Languages (DSL) with examples explaining how we can represent class diagrams and state diagrams in the form of simple DSLs. Finally, an introduction to Gherkin language structure is briefly explained.
2.1 Representation State Transfer (REST)
REST or Representational State Transfer is an architectural style proposed by Fielding [1]
that consists of a set of constraints. A REST service is a web of interconnected resources, based on a hypermedia model that determines possible resource state transitions along with the relationships between the resources. The clients discover which controls to execute at runtime. This constraint is named as HATEOAS [Hypermedia As The Engine Of Applica-tion State]. As a result, it is possible to provide a finite set of valid URIs to the web services as their entry points [3]. The services built on these constraints have a Resource-Oriented-Architecture (ROA) [4]. In order to design and test a ReSTful API, it is important to find the resources and their relationships so that uniform operations can be selected for each resource, and can define the data formats for them.
Before knowing the principles behind REST, it is required to understand the various com-ponents used in a REST API.
Http request and HTTP response
In a RESTful system, the clients and servers interact by sending messages across each other following a predefined protocol. In the case of web APIs, this protocol is HTTP (HyperText Transfer Protocol). The client sends an HTTP request to the server and the server responds with an HTTP response.
HTTP is a request-response based protocol. The client initiates the communication by send-ing an HTTP request and the server will respond back with HTTP response. The structure of HTTP request and HTTP response is explained below.
HTTP request has three main components [5].
Request Method, URI and Protocol Version – This constitutes the first line of the request. It contains the HTTP Request Method, followed by the URI to the method, and the HTTP protocol name with the version used.
Request Header – It is used to communicate information regarding the client envi-ronment. Some of the common headers are Content-Type, Content-Length, Host etc.
Request Body – This is the actual request which is being sent to the server. The headers and body are separated by a blank line. In our context, for REST, the request body is sent in the form of a JSON string.
Similarly, Http response also has three main components [5] which are as follows.
8
Protocol Version, Status Code and Short description – The first line of a HTTP re-sponse contains the protocol name and version, the status code of the request and a short description of the status code. A status code 200 would mean that the request is successful and the description would be ‘OK’. A status 404 would mean that the request was not found and the description would be ‘Not Found’.
Response Headers – These are similar to request headers, except that request header would provide the client environment information while the response header gives the server environment information. For example, Content-Type informs the client how to understand the response body.
Response Body – This is the actual response which is rendered on the client window.
Similar to the request body, in our context, the request body has a JSON structure.
Principles of REST
According to Fielding [1], REST is based on four principles which are as follows.
Principle 1: Resource identification through URI – The first and main principle of REST to think in terms of resources rather than physical files. These resources are identified using URIs which are used for the discovery of resources and their corresponding services.
Some examples of resources with URI are:
www.example.com/image/image.jpg (Image resource)
www.example.com/customers/10001 (Dynamically pulled resource)
www.example.com/videos/v10001 (Video resource)
www.example.com/home.html (Static resource)
Principle 2: Uniform interface – It says to keep the interfaces uniform and simple. This can be achieved by combining the uniform methods of HTTP protocol with the resource operation. The various HTTP methods are GET (getting a resource), POST (creates or sub-mits the resource), PUT (updates the resource) and DELETE (deletes the resource). By com-bining the HTTP methods with the resource names, we can create uniform interfaces leading to simplified communication. The principle is illustrated in Table 2.1.
Ordinary method names HTTP methods REST uniform URL
createPurchaseOrder POST rest/pos
getPurchaseOrders GET rest/pos
getPurchaseOrder GET rest/pos/10001
updatePurchaseOrder PUT rest/pos/10001
removePurchaseOrder DELETE rest/pos/10001
Table 2.1 Sample method names, HTTP methods and URLs
Principle 3: Self-descriptive messages – The metadata of the resources are used and there-fore, the resources are decoupled from their representation and their content can be accessed
9
in various formats. To denote the request and the response, some kind of representation is used, which is JSON in our context. For example, below shows a simple JSON snippet for creating a new plant with name, description and price.
{
"name":"Mini excavator",
"description":"Excavator 1.5 tons", "price":100.0
}
A successful creation results in a response body like the following with HTTP status code 201 (created) and a generated id value.
{
"id":1,
"name":"Mini excavator",
"description":"Excavator 1.5 tons", "price":100.0
}
Principle 4: Stateful interactions through hyperlinks – The interactions with the re-sources are stateless. The interactions are based on explicit state transfer. Every request is independent. The server need not keep track of the previous requests.
2.2 Test-Driven Development
Test-driven development is a programming practise that lets the developers to write addi-tional code only when an automated test had failed and to eliminate duplication [6]. The actual goal of TDD is to write working clean code. The conventional application develop-ment cycle performs coding first, then testing and finally commit. Developers following the TDD approach make an effective adjustment. They perform testing first, coding second and commit. The process is repeated multiple times till the corresponding tests are passing. In other words, the application design is driven by the test.
TDD approach also aims at eliminating duplication. In other words, the written code should not only be testable but it should also be maintainable [6]. Once the test is passed, effort should be put into refactoring the code. Eliminating duplication results in increased cohesion decreased dependency, which are the core aspects of a maintainable code. The biggest ad-vantage of using TDD approach is that a well-structured and test backed code is easier and safer to change.
We will consider an example of Purchase Order - the implementation of the creation of Purchase Order using TDD approach. Initially, we write the test case for the creation of PurchaseOrder. The JUnit test case of the functionality would look like follows.
@Test
public void testCreatePurchaseOrder() throws Exception { Plant plant = new Plant();
plant.set_id(new Long(1));
PurchaseOrder purchaseOrder = new PurchaseOrder();
purchaseOrder.setPlant(plant);
10
purchaseOrder.setStartDate(LocalDate.of(2016, 05, 01));
purchaseOrder.setStartDate(LocalDate.of(2016, 05, 04));
MvcResult result = mockMvc.perform(post("/rest/pos")
.content(mapper.writeValueAsString(purchaseOrder)) .contentType(MediaType.APPLICATION_JSON))
.andExpect(status().is(201)) .andReturn();
purchaseOrder = mapper.readValue(result.getResponse()
.getContentAsString(), PurchaseOrder.class);
Assert.assertThat(purchaseOrder.getPoStatus(), equalTo(POStatus.PENDING));
}
On executing this test case, we would get the following result in Eclipse which implies the test has failed.
The test has failed because the method corresponding to POST /rest/pos is not found.
As a result, we would provide the basic structure of the method.
@RequestMapping(value="/pos",method=RequestMethod.POST) public ResponseEntity<PurchaseOrder> createPO(
@RequestBody PurchaseOrder po) {
HttpHeaders headers = new HttpHeaders();
return new ResponseEntity<PurchaseOrder>(po, headers, HttpStatus.valueOf(201));
}
We execute the test again. Again the test is failed due to assertion error saying that it ex-pected a status of “PENDING” but obtained was null. As a result, we provide the complete implementation of the creation of PurchaseOrder like follows.
@RequestMapping(value="/pos",method=RequestMethod.POST) public ResponseEntity<PurchaseOrder> createPO(
@RequestBody PurchaseOrder po) { po.setPoStatus(POStatus.PENDING);
po = purchaseOrderRepo.save(po);
HttpHeaders headers = new HttpHeaders();
return new ResponseEntity<PurchaseOrder>(po, headers, HttpStatus.valueOf(201));
}
The tests are run again.
11
As seen above, we got the tests passing. To summarize, we implemented the creation of Purchase Order functionality using TDD approach by writing a failing test initially and then by implementing the functionality by repetitive development and testing process.
2.3 Class diagram and State chart diagram
A class diagram is a UML static structure diagram that describes an application using its classes, attributes and operations, and the relationships between those classes. A class in a class diagram is represented by a box with three rectangular boxes inside it. The top box provides the name of the class, the middle rectangle contains the attributes in the class and the lower box contains the operations declared within the class. The classes will be related or associated to other classes, which are depicted by lines between them.
Figure 2.1: Class diagram example
Figure 2.1 above shows a basic class diagram of a Purchase Order scenario. The class dia-gram shows two classes (PurchaseOrder and Plant) and one enumeration (POStatus). It shows the various relationships that exist between the classes. For example, Plant has a
“contains” relationship with PurchaseOrder which is a one-to-one relationship. The POSta-tus enumeration provides the various staPOSta-tuses of PurchaseOrder.
A state chart diagram is a representation of a state machine that visualises the change of states of a modelled element. It shows which activities are executed based on the occur-rences of events. It displays the various states an object goes through in its life, based on an
12
external event [7]. States, events and transitions constitute a state diagram. States are those values which certain attributes of an object possess. An object continues to be in a single state until an event is triggered on it. An event is any kind of occurrence applied to the object which may or may not change its state. The same event can have multiple effects for various states. A relationship between two states is termed as a transition which indicates a change from a state to another state [8].
Figure 2.2: Statechart diagram
The statechart diagram for a Purchase Order is shown in Figure 2.2. The state diagram dis-plays various transitions Purchase Order could go through from one state to another. For example, if the Purchase Order is in “Pending” state, it can either go to “Open” or “Rejected”
states. When the clerk calls acceptPO method, the status would become “Open” and if he calls rejectPO method, then the status would be “Rejected”.
2.4 Domain Specific Language (DSL)
A domain specific language (DSL) is a small programming language or execution specific language which provides a notation towards the application domain and is focused on some concepts and features of that particular domain [9]. The domain can be anything. SQL and XMLs are examples of domain specific languages. A DSL can be used to generate various contents of a system in an application domain. A well-designed DSL is based on a clear understanding of the underlying application domain so that the required contents can be generated easily [9]. One example of textual DSL representation of the class diagram in Figure 2.1 is represented below.
13
entity PurchaseOrder { plant: Plant
Similarly, the state chart diagram in Figure 2.2 can be denoted in the form of DSLs. One such example is provided below.
events
Since a DSL can take any structure, we can model it as a Gherkin feature and provide the characteristics of a state chart model and other details in the feature file.
2.5 Gherkin language
We consider Gherkin language as the DSL in the approach as a Gherkin feature can be used to represent the resource associations and the corresponding state transitions. Also, it is ideal to use an existing language rather than generate a completely new DSL as it can be reused for other purposes as well.
14
Gherkin is the language used for writing Cucumber features. The biggest advantage of using Gherkin is that it is readable not only by the computer but also the stakeholders as it is written using plain spoken language [10]. The keywords used in Gherkin has been translated into over forty languages. It is not tied down to any particular spoken language. As a result, we can say that even though it is considered as a programming language, its primary goal is human readability. It means that we can write automated tests that can be read like a docu-mentation. A small Gherkin example is given below.
Feature: PurchaseOrder feature As a customer
In order to rent plant equipment I need to process a Purchase Order Scenario: Creation of Purchase Order
When customer submits po Then po is saved in database And customer is notified
Scenario: Accepting a Purchase Order When clerk accepts Purchase Order Then Purchase Order is accepted
A Gherkin file uses .feature file extension. Every file starts with the Feature key-word. It is followed by a text which is the name of the feature and the lines below them is the description or narrative. Any text can be provided in the description except a line starting with one of the keywords used in the language. Gherkin keywords are as follows.
Feature Background Scenario Given
When Then And But
* Scenario Outline Examples
Here, the main keywords we would cover are Background, Scenario, Scenario Outline and Examples. The keywords Given, And, But, When and Then are used to start a step within the other keywords. The behaviour of the application is described in a feature file using the scenarios. Each scenario provides a concrete example of how the application should respond in a particular situation. Adding together all the scenarios would describe the expected be-haviour of the feature. In Gherkin, we use mainly Given, When and Then keywords to identify different parts of a scenario. The Given keyword set up the context where the scenario is executed, When to start interacting with the system and Then to check if the expected result is the same as the outcome of the interaction. And and But keywords are used to add more steps to the above three keywords.
A simple example of a scenario in a Gherkin feature file is given below.
Scenario: Accept plant request
Given the plant status is 'PENDING'
When customer receives request to accept And customer accepts the plant
Then status becomes 'ACCEPTED'
In the above scenario, we can see that the And keyword is used to add an additional When step. Similarly, we can use the But keyword to extend the functionalities of each of the three step keywords.
15
The background section provides a set of steps common to each scenario in the file. We use them in order to avoid repetition of steps in each scenario. Let us consider the following example where we define the initial database before the execution of each scenario.
Scenario: Accept plant request Given the following plants
| _id | name | description | price | status |
| 1 | Plant1 | Excavator 1.5 tons | 100.00 | PENDING | When customer accepts the plant with _id '1'
Then status becomes 'ACCEPTED'
Then status becomes 'ACCEPTED'