5.2 Improvement for Development Process
5.2.4 Results
Questionnaire
After the presentation of inclusive design issue and each software front end, ques-tions are raised.
The questions relate to ISO Norm 9241-110 (see Schneider [2008]) with four cate-gories according to the following themes:
∙ Suitability for the task
Does the software help designers to complete a task of product design for elderly people without burden?
∙ Self-descriptiveness
Each software tool explained sufficiently and comprehensive?
∙ Conformity with the expectations
Is the structure of the software ok and does it conform with habits of designers?
∙ Suitability for learning
Do the tools require pre knowledge?
General Questions
Table5.2 presents the results of the general questions about pre knowledge of each participant. 8 participants describe themselves as familiar or very familiar about de-sign of physical products (8 / 11 ∼72%). As mentioned in 5.2.3, mostly designers were questioned but also three researchers in the field of design participated1. The reason was to get a more resourceful perspective on the results from practical but also theoretical point of view on design and ergonomic factors.
The answers of CAD knowledge are due to the fact not all designers work with a virtual environment but rather with sketch drafts or other tools. All participants are at least slightly familiar with CAD. Both front ends were evaluated by each experience level of designers.
Although not all participants were experienced in CAD, a strong familiarity with in-clusive design was there.
All participants were at least slightly familiar withVirtual User Models. The group of participants of this evaluation is thus appropriate.
All described methods of customer involvement were applied:
∙ Quality function deployment (QFD) 7/112
∙ User-oriented product development 1/11
∙ Concept testing 3/11
∙ Beta testing 3/11
∙ Consumer idealized design 1/11
∙ Lead user method 3/11
∙ Participatory ergonomics 1/11
As expected, QFD is the most used method (see also section 2.4 or Akao [2004]).
The group of participants is suitable as the modification of the product development process does have a strong impact on this method (see section 4.3). Both Concept testing and beta testing were used by the same amount, one participant described both together. Lead user method, concept and beta testing were applied in combina-tion with QFD, as they do not cover the completeproduct development process.
1Answers only differ in applied customer involvement methods as researchers selected none.
23/11 participants from research selected none applied customer involvement method.
The coverage of involvement methods is positive with respect to the inclusion of the software framework into differentproduct development processes. Each method was applied by at least one participant of this study.
Suitability for the task
Statement - - - +/- + ++ skip
”The Sketch Application provides a wide choice of scenarios.”
0 2 5 2 2 0
”The design recommendations of the toolset are necessary.”
0 0 0 5 6 0
”It takes a short time to go through recommendation list.”
2 1 3 3 2 0
”I would need a user manual to use the software.”
0 1 3 6 1 0
”The look and feel of the application was suitable and pleasant.”
0 2 3 4 2 0
”The software is easy to use.” 0 2 0 8 0 1
Table 5.3: Results of questions regarding suitability for the task
The suitability of the software framework to be added into existingproduct devel-opment processesis covered in this section. Table5.3presents these results.
The first statement is questioning if the choice of scenarios of user model, typ-ical environment in which the product would be used and typtyp-ical task performed using the product is sufficient enough. The result is slightly positive but ambigu-ous. Almost half of participants answered neutrally (5/11 answered “+/-”), infers that the current system describes a good base but can (and probably should) also be extended.
The necessity of design recommendations is throughout answered very pos-itive. All participants agreed, resulting in the conclusion that the concept of a recommendation-driven product development process is not just accepted but also needed. Although the customer involvement method applied by each participant is very different, the inclusion of recommendations is advantageous independently of involvement method applied.
The variety of approval about the time consumption to go through the recommen-dation list can be seen very ambiguous (see figure 7). While on the one side it would be time and cost saving to be able to go through the recommendations very fast, on
the other side a deceleration of product design would be more fruitful with respect to more accessible products and designer creativity. The aim of this question was to maintain this assumption. This will also be an issue in the next section of the evaluation.
The next two statements were answered very similarly with a trend to approval (figure 8and9). Participants of the study would need a user manual to the software and the look and feel of the software was suitable and pleasant. Regarding the front end presentation, the results are slightly positive but can also be improved.
The last question of this section described the approval to the general statement that the software is easy to use. Eight participants approved this (“+” bar in table 5.3), while 2 disapproved (“-”) and one skipped this question. The strong approval concludes that the software in its current state is already easy to use by designers but can be improved. One participant could not use the software directly on the own pc and commented that he was not able to install and use the software on a Mac OS.
Self-Descriptiveness
Statement - - - +/- + ++ skip
”The description of information in the sketch application for user profiles is comprehensible.”
0 1 1 6 3 0
”The description of information in the sketch application for
environments is comprehensible.”
0 1 1 4 3 2
”The description of information in the sketch application for tasks is comprehensible.”
0 1 1 7 2 0
”The description about recommendations is comprehensible.”
0 1 0 7 3 0
Table 5.4: Results of questions regarding self-descriptiveness
Information on user profiles, environments, tasks and recommendations is col-lected next (see table 5.4). The results show a trend towards approval (most partic-ipants answered “+”), concluding a general approval but also some ambiguities with respect to comprehension of each model. More information about the scenario would be advantageous. As presented in figure5.3part (A), meta information about the cur-rent scenario selection can be seen in the software. For instance impairment levels
for each user model instance are presented, but designers do not exactly understand the meaning. An improvement of the scenario explanation would be beneficial.
Conformity with user expectations
Statement - - - +/- + ++ skip
”The software has a consistent structure.”
0 0 3 7 1 0
”The layout was as expected.” 0 1 2 6 1 1
”Some features of applications do not have an unpredictable
processing time (e.g. start of application).”
0 0 2 6 0 3
Table 5.5: Results of questions regarding conformity with user expecta-tions
The structure of the software was mostly approved by participants as seen in table 5.5. The layout was similar to expectations of designers. This leads to the assumption of a positive conformity with user expectations. The last statement, that some features of applications do not have an unpredictable processing time was slightly approved. Some participants skipped this question, because they could not find any features with unpredictable processing time, so they were unclear about the result. Eight participants found some. For instance the start of the application takes some time which is unclear from designers perspective. As previously mentioned in chapter 4, during the start the reasoning is performed from an initial Ontology inferring a new resulting model which is used in the application. In the current state, the model is inferred dynamically by every start of the application regarding further implementations and extensions of the model itself.
Suitability for learning
Statement - - - +/- + ++ skip
”The software requires little time to learn.”
0 1 5 4 1 0
”The software is easy to learn without prior knowledge, help or manual.”
1 1 2 7 0 0
”The software is easy to use, even without having prior knowledge.”
0 3 2 6 0 0
Table 5.6: Results of questions regarding conformity with user expecta-tions
The first statement asks how time consuming it is to learn how to use the soft-ware (see table 5.6). In average the result of this statement was slightly positive among participants, concluding that it takes some time to learn. The next both statements consolidate this statement, regarding the software is easy to learn with-out prior knowledge, help or manual with a strong agree by participants. This goes in line with the conclusion of results that a user manual would be advantageous.