Purchasing Auctions – A Synthesis of Current Research

(1)

Purchasing Auctions – A Synthesis of Current Research

by

Thomas Germer

Ph.D. Student and Research Associate

WHU Otto Beisheim Graduate School of Management Herbert Quandt Endowed Chair for International Management

Burgplatz 2, Vallendar 56179, Germany Phone ++ 49.261.650.9323

Fax ++ 49.261.650.9329 tgermer@whu.edu

Craig R. Carter, Ph.D.

Assistant Professor of Supply Chain Management University of Nevada

College of Business Administration Reno 89557-0206, U.S.A.

Phone ++ 1.775.784.6993, Ext. 321 Fax ++ 1.775.201.1754

crcarter@unr.edu

Lutz Kaufmann, Ph.D.

Professor and The Herbert Quandt Endowed Chair in International Management WHU Otto Beisheim Graduate School of Management

Burgplatz 2, Vallendar 56179, Germany Phone ++ 49.261.650.9320

Fax ++ 49.261.650.9329 kaufmann@whu.edu

Published in:

Proceedings of The 15th Annual North American Research Symposium on Purchasing and Supply Management, Tempe,

AZ, USA, 2004, pp. 119-139

(2)

1. Introduction

This paper contributes to the current research on online reverse auctions in industrial purchasing by presenting a detailed literature review on purchasing auctions. To the best of our knowledge, such an extensive literature review has not yet been published.

Despite some minor semantic differences, auctions are commonly defined as ‘a market institution with an explicit set of rules determining resource allocation and prices on the basis of bids from market participants’ (McAfee and McMillan (1987) p. 701). Only a few earlier publications limit the auction use to portable goods (see Niessen (1974) p. 11, and Rathgen (1911) p. 278) or require the physical presence of the auction item (see Niessen (1974) p. 11).

Consequently, electronic (or online) auctions can be considered as a market institution with an explicit set of rules determining resource allocation and prices on the basis of electronically submitted bids from market participants. As some researchers narrow the definition of online reverse auctions to a downward price format (e.g. Emiliani and Stec (2002a) p. 12) or to bids submitted only via the Internet (e.g. Pearcy, Giunipero and Dandeo (2002) p. 328), we determined that a broader definition was appropriate, in order to also encompass purchasing auctions in a dutch format (where prices rise) and auctions where bids are submitted electronically, but not via the Internet (e.g. via an EDI access, see Bichler (2001) pp. 15, and Amor (2000) pp. 29).

Auctions are widely used as a purchasing tool in industrial, governmental and personal environments. Since auctions are among the oldest transaction mechanisms dating back to 500 B.C. (see Milgrom and Weber (1982) p. 1089), an impressive amount of articles on auctions has been published to date. Already in 1980, Engelbrecht-Wiggans ((1980) p. 119) reported more than 500 papers on auctions. In the summer of 2003, the EbscoHost database listed more than 17,400 non-refereed and about 1,400 refereed articles on auctions. Much of the auction research is fragmentary and varies widely in scope (see Riley and Samuelson (1981) p. 381). As auctions initially became popular in microeconomic theory, especially in the form of the invisible

‘Walrasian Auctioneer’ ensuring a market equilibrium, auction research was and still is predominantly limited to issues in economics. If covered at all, purchasing as a managerial task is usually only a side-issue in these papers.

Consequently, the literature on auctions published to date is in need of a critical review.

A review provides a basis for both theoretical and empirical research on purchasing auctions.

This paper provides such a review, and is organized as follows: Section 2 gives an introduction to the methodology for literature reviews. Section 3 shows our specific search criteria to find relevant articles on auctions in purchasing. These relevant papers are then thoroughly analyzed in terms of content (section 4) and methodology (section 5). The paper concludes in section 6 with a summary and an agenda to help guide future research.

(3)

2. Methodology for Literature Reviews

A literature review in general can be seen as a ‘systematic, explicit, and reproducible method for identifying, evaluating, and interpreting the existing body of recorded work produced by researchers, scholars, and practitioners’ (Fink (1998) p. 3). Reproducibility is an important feature because different search techniques lead to different samples of studies (see Cooper (1998) p. 41).

According to Cooper, scientific information can be gathered from primary and secondary channels (see Cooper (1998) pp. 42-77). Primary channels offer direct access to research results, whereas secondary channels only provide information about research results. Primary channels can contain both informal sources (like personal contacts, the world-wide-web, etc.) and particularly formal sources, like electronic journals or reference lists of published studies.

Secondary channels are established and administered by third parties in order to provide an extensive body of relevant publications to researchers – this clear mission is a major difference from the world-wide-web. Research bibliographies, prospective research registers (e.g. in medical science), citation indexes, and reference databases are the most important secondary channels. In consideration of reproducibility as an important feature of a literature review, publicly accessible reference databases and research registers form the backbone of any comprehensive literature search (see Cooper (1998) p. 74, and Fink (1998) p. 18).

When using reference databases, two important limitations have to be considered. First, the time lag between publication and appearance in the database can be quite long. Second, each database usually employs certain rules of acceptance in terms of content and quality (see Cooper (1998) pp. 60-61). Publications, in particular, which cover research on two commonly separated disciplines (e.g. economics and psychology) are likely to be kept in the databases of only one discipline. Thus database queries can provide incomplete results. This is why Cooper suggests to additionally incorporate articles to the literature review that were found randomly / unsystematically (see Cooper (1998) p. 75). For Fink, the use of additional channels comes with certain strings attached (see Fink (1998) p. 34):

• The topic is new and its associated concepts have not yet been incorporated into official subjects’ headings (likewise Cooper (1998) p. 61).

• Search terms are used inconsistently because definitions in the field are not uniform.

• There is reason to believe that many important studies are in progress or completed but not published.

• Access to on-line databases is limited.

Besides the last aspect, the strings formulated by Fink result directly or indirectly from the novelty of a topic which applies to the research of online reverse auctions, too: Both the practical use and the scientific discussion on online reverse auctions in purchasing are in an early stage. Therefore it is justifiable to include further references in the literature review that were not found systematically.

(4)

3. Search Terms and Data Sources

In order to present the current state of research on purchasing auctions, we conducted an extensive literature review covering peer-reviewed articles and monographs in the English and German languages. The search was limited to those two languages due to language restrictions of the authors as well as a matter of research efficiency. To expand upon this current state of research on purchasing auctions, two approaches are suitable. The research can be focused on either the specific application of auctions for purchasing purposes or the electronic handling of the auction mechanism. From our point of view, the specific application of auctions to purchasing is the dominating feature, because it is the prerequisite for the discussion about the electronic support of an auction. This distinction is by no means philosophical, but relevant for the literature search. After an initial examination of the literature, we realized that articles on electronic auctions in purchasing are basically a sub-category of the extensive body of literature on electronic commerce, not of the literature on purchasing management. In order to avoid any undesigned exclusion of relevant literature, we therefore included ‘electronic’ in our set of search terms.

This paper adopts the view of the purchasing organization. Nevertheless, we decided not to limit our search for relevant literature to this perspective for three reasons. First, terms like

‘selling auction’ or ‘purchasing auction’ do not give a clear and unique description of the type of auction they refer to, because every auction has by definition buyer(s) and seller(s). Hence, there is a purchasing perspective in every auction. Second, the distinction between selling auctions and purchasing auctions is increasingly neglected. Even though auctions were historically interpreted from a selling perspective (see Milgrom and Weber (1982) p. 1090 as an example), in many recent publications the term auction is also used to describe ‘both bidding to buy and offering to sell’ (Wei (1997) p. 65, and McAfee and McMillan (1987) p. 701). Third, an auction designed to purchase a good or service can also be interpreted as an auction intended to sell the right to supply a certain good or service to the demanding institution.

We included only articles and monographs in the English or German languages that were listed in publicly accessible databases (see Fink (1998) pp. 52-55). We did not exclude any particular journal up-front, but examined only articles published in peer-reviewed journals (for monographs, a peer review is not common). A peer review is an important quality indication for scientific research (see Cooper (1998) p. 79, and Schlinghoff and Backes-Gellner (2002) p. 347).

Some of the articles found unsystematically were not peer-reviewed.

We used EBSCOhost® database to search for English articles. EBSCOhost® belongs to Ipswich-based EBSCO Publishing, ‘the world’s most prolific aggregator of full text journals, magazines and other sources’ (EBSCO (2003)). We employed the ‘standard search’ mask with the search terms ‘electronic auction’, ‘reverse auction’, ‘procurement auction’ and ‘purchasing auction’ to find peer reviewed articles. All available data sources (Business Source Premier, Econlit and Regional Business News) were included in the search without time restrictions. The search was completed on July 20, 2003 and resulted in 43 publications.

(5)

For German publications, we used the ‘WISO-Net Wiwi’ database that keeps more than 1.9 million references to German and international scientific journals, books and ‘grey literature’

(e.g. working papers) in economics, management, the financial service sector, and ergonomics (see WISO (2003)). As with the English references, we employed the analogue German search terms ‘Auktion’, ‘Elektronische Auktion’ and ‘Online Auktion’ and the two search term combinations ‘Auktion’ plus ‘Beschaffung’ (= purchasing) and ‘Auktion’ plus ‘Einkauf’ (=

procurement) (‘plus’ in the sense of an intersection, not in the sense of a set union) without any time limits. The search was completed September 15, 2003 and resulted in five German and two additional English publications. Obviously irrelevant articles, e.g. book reviews or reports on stamp auctions were eliminated from the gross results and not included in our review. In addition to the articles we found systematically with the search techniques described above (marked with a ‘s’ in Figures 1 and 2), we included 13 papers that were published in conference proceedings and in non-reviewed journals in our literature review (marked with an ‘u’ for ‘unsystematically found’). We regarded these articles to be relevant for the status quo of the topic due to the title of the publication. Over all, we analyzed 63 publications.

4. Key Findings regarding Content

All publications were analyzed in terms of content and methodology (see Croom et al.

(2000) pp. 71). Regarding content, we first examined the parties participating in the auction (companies, government or individuals) and their specific role. Next, we looked at whether the publication deals predominantly with either the auction process or the conducting institutions.

We further considered whether the article was written from a purchasing or a selling perspective.

Finally, we thoroughly reviewed all references to see if managerial aspects in purchasing were covered.

Remarkably, auctions in purchasing are a relatively new topic in academia – regardless of the Internet hype. The first paper by Luton and McAfee dates back to 1986 and covers sequential procurement auctions from a governmental perspective. As of the year 2000, only 14 papers on auctions in purchasing were published. But since 2000, 49 publications were released: seven in 2000, nine in 2001, 22 in 2002, 10 in 2003, and one in 2004 (at the deadline of this review).

These numbers emphasise the growing practical and scientific relevance of auctions in purchasing, obviously amplified by the enormous popularity of Internet auctions.

This paper deals with auctions between companies. Our review in terms of content starts with the parties participating in an auction as buyer(s) and supplier(s). If both buyer(s) and supplier(s) are companies, we use the term B-2-B auction (Business to Business). If governmental or public institutions acts as a buyer and companies as supplier(s), we term these auctions B-2-G (Business to Government). Auctions on private platforms like eBay take place between two private individuals (C-2-C – Consumer to Consumer) or between companies as sellers and private individuals as buyers (B-2-C – Business to Consumer). Our review of the articles indicates that auctions in an industrial environment (B-2-B) were basically not covered before 2000. Besides the articles by Hallwood (1996), Wolfram (1998) and Teich, Wallenius and

(6)

Wallenius (1999), all articles published until 2000 dealt with auctions in a governmental environment or covered no institutional aspects at all (purely conceptual). Over all, only 51% (32 of 63) of the literature reviewed covered procurement auctions in a B-2-B context. About 29%

(18 of 63 publications) covered auctions in a governmental environment; roughly 16% (10 of 63) do not cover any or at least no significant institutional aspects. The remaining 4% covered auctions with private consumers.

As this paper addresses both the auction process and the participating institutions, we analyzed to what extent both aspects are covered in the current literature (see also Jap (2003) p.

98). Process-related in this context means that those papers predominantly deal with the design, influencing factors and optimization of an auction process. Wolfstetter (1998) for example analyzes the design of the auction process in the case where the value of the auction item is correlated among buyers and sellers. Oh (2002) compares the likelihood of a ‘winner’s curse’ in a B-2-C and a C-2-C auction. We found that more than 52% (33 of 63) of the articles primarily discuss such process-related issues. Only 35% (22 of 63 publications) cover both process-related and institutional aspects. Institutional aspects are all intraorganizational issues that occur within one or more participant’s organization in connection with an auction event. Beckmann (1999) for example discusses the relation between auctioneer and seller as well as the relation between auctioneer and bidder. Wilcox (2000) and Elgart (2001) do indeed cover institutional aspects, but only for private consumer and public institutions. Emiliani (2000), Hong and Hartley (2001), Jap (2002), and Smeltzer and Carr (2002) are among the first authors explicitly covering the purchasing company’s perspective. Generally, the articles on auctions are predominantly process-related.

Next, we surveyed the perspective from which the purchasing auction was analyzed. We found seven publications mainly analyzing auctions from the view of the selling institution; 11 articles adopt more or less a neutral position. These results strongly support our earlier assertion that literature on purchasing auctions cannot automatically be considered as purchasing literature. These findings further underline the fact that the coverage of auctions in purchasing is actually significantly lower than the number of publications found initially indicated.

Finally, we examined all publications to determine whether managerial aspects of purchasing were analyzed. As opposed to interorganizational aspects in the previous question, these aspects refer to intraorganizational issues, such as the integration of auctions into the purchasing toolset or the impact of auctions on purchasing processes. Kinney (co-founder of FreeMarkets.com) and Emiliani annotated selected issues from a purchasing perspective in a more or less anecdotal manner already in 2000 (see Kinney (2000) pp. 41 and Emiliani (2000) pp. 181). Emiliani and Stec (2001, 2002a and 2002b), Hong and Hartley (2001), Jap (2002), and Pearcy, Giunipero and Dandeo (2002) were among the first authors who integrated managerial issues related to the use of auctions in purchasing.

Jap (2002) is one of the first authors who explicitly included the effects of purchasing strategy on auction design and the financial and non-financial benefits of the use of an auction (see Jap (2002) pp. 513). In addition to these two issues, Beall et al. (2003) extensively further

(7)

address managerial issues, including the cross-functional formation of auction teams (see Beall et al. (2003) pp. 44). Analyzing the food company Mars Inc. as an example, Hohner et al. (2003) examine the internal problems of purchasing management and their implications for the specific design of auctions and the auction process. The publication by Luedtke (2003) is one of the first holistic discussions on purchasing auctions in German speaking countries from a managerial perspective. Based upon case studies and expert interviews, Luedtke reveals influencing factors on the degree of utilization of auctions in purchasing, discusses design parameters, and proposes a set of five auction modules. Furthermore, the phenomenon has so far not been examined in cross-national contexts (exceptions are Beall et al. (2003) and Kaufmann and Carter (2003)).

Figure 1 provides a brief summary of the content of all of the articles reviewed in our paper.

5. Key Findings regarding Methodology

Our review of the research methodology employed in the existing literature specifically focused on the degree of empirical support for the papers’ findings and assertions. Over all, the status quo of empirical research on electronic auctions in purchasing is significantly low. Only 43% (27 of 63) of all publications contain some empirical data at all. The articles only reproducing existing case studies (e.g. Bajari and Summers (2002)), quoting company examples without rigorous analysis of data (e.g. Kannan and Kopalle (2001)) or announcing a forthcoming survey (e.g. Hong and Hartley (2001)) were not considered ‘empirical’.

Looking at the data collection process, close to 50% of all empirically supported literature (13 of 27) was based on secondary data. For example, three articles employ databases from several Departments of Transportation (see Porter and Zona (1993), Gupta (2002), and Hong and Shum (2002)). Another three papers made use of eBay’s database (see Wilcox (2000), MacDonald and Slawson (2002), and Ottaway, Brunneau and Evans (2003)). A strong advantage of the use of secondary data to test hypotheses is that a large amount of data is often available for statistical analysis. Besides Sareen (1999), all authors working with secondary data use samples larger than 100. Not surprisingly, then, all 13 publications working with secondary data employ multivariate data analysis.

Empirical evidence for B-2-B auctions was provided in only 16 publications, with four based upon secondary data (Wolfram (1998), Koppius and van Heck (2002), Smart and Harrison (2002), and Bajari, McMillan and Tadelis (2003)) and 12 upon primary data. To the best of our knowledge, the study by Jap (2002) is the first to gather primary empirical data by a large-scale survey and provide multivariate data analysis (exploratory factor analysis). In addition to the work by Jap (2002 and 2003), only Pearcy, Giunipero and Dandeo (2003) gather empirical data on online reverse auctions with a large-scale survey. All other empirically supported publications employ case studies to gain empirical insights.

The case study method can be used for several purposes, including building theories (see Eisenhardt (1989), Glaser and Strauss (1967)), deepening the insights from a large-scale survey (see Ellram (1996) p. 97), and providing teaching material (see Yin (2003)). Compared to a

(8)

large-scale survey, case studies are preferable if the status quo of (empirical) research in a specific area is in a very early stage (see Yin (2003) pp. 3). This is the case for the research on auctions in purchasing. With this in mind, the current status of empirical studies in purchasing auctions is a logical consequence of the novelty of this research area. We assume that the amount of large-scale surveys on auctions in purchasing will rise significantly in the near future. Figure 2 abstracts the methodology of all reviewed publications.

6. Summary and Agenda for Future Research

We analyzed 63 different articles and monographs in our literature review dealing with purchasing auctions in some way and examined these articles in terms of content and methodology. In a nutshell, the reviewed articles predominantly cover transactional aspects and their mathematical optimization (how to auction). Issues regarding the appropriate use of auctions in purchasing (when to auction) and the integration of auctions into the purchasing tool set are not discussed in most articles.

The reviewed articles seldom focus on purely industrial transactions (B-2-B) but mainly on transactions between companies and the government (B-2-G), between companies and private consumers (B-2-C) or on purely private auctions (C-2-C) (see Hong and Hartley (2001) p. 83).

Regarding methodology, the majority of the articles employ highly formalized mathematical optimization terms and are presumably not plausible/directly applicable for purchasing managers. Empirical support of theoretical concepts is rarely provided. If so, empirical data is predominantly derived from secondary channels or only one or more case studies and not from a large-scale survey. Figure 3 provides a synopsis of the current literature on auctions in purchasing in terms of content and methodology.

Future research should concentrate more on managerial issues. The auction process itself is already heavily researched by theoretical literature in economics (see Jap (2003) p. 98).

Several other issues emerge:

• Additional research is needed to gain evidence on the proposed long-term benefits and concerns of auctions in purchasing (e.g., cycle time reductions, savings or sourcing performance).

• Furthermore, the question of how to integrate auctions appropriately in the purchasing toolset and in an auction-integrated sourcing process will be of vital interest for both academia and practice.

• As auctions are a world-wide phenomenon, regional and cross-national/cross-cultural studies examining similarities and differences might be of keen interest especially for those companies employing an international and decentralized purchasing organization.

Beall et al. (2003) and Kaufmann and Carter (2003) conducted such research with the only cross-national comparisons of auction practices published so far. The qualitative research carried out on auctions so far has laid the groundwork for further research particularly with large-scale surveys and multivariate statistics.

(9)

Source Year of

Publication Author(s) Participants Focus on process and/or institution Perspective

Covered managerial aspects in purchasing?

Abstract

s 1986 Luton / McAfee B-2-G process purchasing no Formulation and discussion of a formal model about sequential procurement auctions with the government as the purchasing institution.

s 1989 Dasgupto / Spulber B-2-G process purchasing no

Presents an auction model that allows participating companies to vary quantities and technical specifications of the auction item. In addition to the common model with fixed quantities and qualities, demand now depends on the costs of the lowest bidder.

s 1992 Anton / Yao B-2-G process purchasing no

Analyses and discusses a 'winner takes all'- market compared to 'split award'-auctions with complete and incomplete information in terms of price.

s 1993 Porter / Zona B-2-G process purchasing no Formulation and empirical test of a model to identify collusive behavior in governmental auctions for road construction work in the U.S.

s 1996 Hallwood B-2-B process purchasing no

Discusses the reason to chose an 'invited tender-bid procurement auction' instead of a regular face-to-face negotiation for oil companies.

Additionally, implications of nondisclosure of prices after an auction for current and future contracts are analyzed.

s 1996 Snyder n/a process purchasing no Develops a model to explain the relevance of purchasing company's

characteristics on the likelihood for collusive behavior among suppliers.

s 1997 Bag n/a process purchasing no

Discusses auction design when suppliers decide to invest in cost- reductions before bidding. This investment may rise the likelihood of winning the auction, but do not automatically improve the competitive situation of a supplier in an auction.

s 1997 Branco n/a process purchasing no Designs a two-stage bidding process for multi-dimensional auctions

when costs are correlated among suppliers.

s 1998 Rodrígez-Aguilar et

al. n/a process n/a no Modifies an agent-based electronic auction system towards a 'test-bed'

for electronic trading agents in internet auctions.

s 1998 Wolfram B-2-B process sales no Formulates and tests a model to test 'strategic bidding' and its

implications for daily electricity auctions in England and Wales.

s 1998 Wolfstetter n/a process sales no

Gives an overview on auctions and their relevance in the economy and presents basics on auction theory. The impact of mutual dependencies of private quality signals between sellers and buyers - known as the 'linkage-principle' - is discussed in brief.

s 1999 Beckmann n/a institution and

process n/a no

Researches the organization of the German market for auctioneers and auction service provider supported by a large-scale survey. Especially the relation between auctioneer and seller and between auctioneer and buyer is analyzed.

s 1999 Sareen B-2-G process purchasing no

Considering 37 auctions of the public Indian oil company IOC as example, this article investigates whether the common value or the independent value paradigm better explains the use of a sealed-bid format for the procurement of oil by employing a 'posterior odds ratio'.

s 1999 Teich / Wallenius /

Wallenius B-2-B/C process n/a no

Describes different algorithms for multidimensional auctions and markets and exemplifies those algorithms for the electronic stock trading program 'OptiMark'.

u 2000 Emiliani B-2-B institution and

process

purchasing

and sales no

Describes auction processes for direct materials in B-2-B relationships and presents benefits and concerns of auction use for both buyer and seller.

s 2000 Jap B-2-B institution and

process purchasing no Brief description of a research project on online reverse auctions carried out by Sandy D. Jap.

s 2000 Kinney B-2-B n/a purchasing no

Anecdotal comparison of the Internet marketplaces 'Bid-Ask Neutral Marketplaces', 'Seller-Bidding Reverse Auctions' and 'Buyer-Bidding Reverse Auctions' and description of a seller-bidding reverse auction process.

s 2000 Kjerstad / Vagstad B-2-G process purchasing no

Discusses the impact of a variable number of bidders on the outcome of an auction. Special focus on the role of entry investments for participating suppliers.

s 2000 Marshall B-2-B institution purchasing no Short introduction on the web portal www.cfoweb.com which can be

used to buy different financial products.

s 2000 Wang B-2-G process purchasing no

Analyzes the market equilibrium and social welfare for auctions with a renegotiations after the actual auction event (comparison to the model by Bulow / Klemperer).

s 2000 Wilcox B-2-C institution and

process n/a no Examines the role of experience on bidding strategies for auctions for four different product categories at eBay.

Bibliography Content

Figure 1: Summary in Terms of Content (1/3)

(10)

Source Year of

Abstract

s 2001 Bajari n/a process n/a no

Studies a model for competitive bidding in purchasing auctions with asymmetric bidders. More realistic models have to consider asymmetric bidders and bids because in reality, different locations, different management skills and collusive behavior leads to asymmetries.

s 2001 Buchwalter B-2-B process purchasing no Analyses the impact of modern IT technologies, esp. the Internet, on purchasing processes (purely conceptual).

s 2001 Elgart B-2-G institution and

process purchasing no

Anecdotal report on the introduction and the usability of electronic reverse auctions for US Army's CECOM (Communications-Electronics Command).

u 2001 Emiliani / Stec B-2-B institution purchasing yes Presents and discusses common contract conditions in industrial reverse auction purchasing processes.

u 2001 Hong / Hartley B-2-B institution and

process purchasing yes

Starts with a brief introduction on reverse auctions and their specific pros and cons. Presents a conceptual model to research buyer's satisfaction with online reverse auctions.

s 2001 Kannan / Kopalle B-2-C process sales no

Illustrates dynamic prices on the Internet and their use. Focuses on 'e- tailers' and perishable goods (airline tickets, hotel accommodations etc.).

s 2001 Oliva B-2-B process sales no

Anecdotal report on opportunities and concerns from online reverse auctions from a seller's perspective under reference to the Harvard Business Review paper by S. D. Jap (2000).

u 2001 Rosenkranz B-2-B institution and

process sales no

Discusses whether two companies should cooperate due to externalities and whether they should bid in an auction as partners or competitors.

s 2001 Sim / Wong n/a process n/a no Presents an agent-based auction system which is able to employ

market-driven negotiation strategies and a contract-optimization tool.

s 2002 Bajari / Summers B-2-G n/a purchasing no

Describes a two-stage test from Bajari and Ye to detect collusive behavior among bidders. If bids in sealed-bid auctions are neither 'conditional independent' nor 'exchangeable', they are most likely to be collusive and not competitive.

s 2002 Compte / Jehiel n/a process n/a no

Analyzes the change in social welfare when additional bidders bid in second price sealed bid auctions and in english auctions. Hypothesises that bidder's valuation of the auction item is influenced by both the common value model and the independent private value model.

s 2002 Davis / Wilson B-2-G n/a purchasing yes (B-2-G) Examines the correlation between cost structure, communication and conspirative behavior for both fixed and endogenous cost structures.

u 2002 Emiliani / Stec B-2-B institution and

Discovers several discrepancies between the 'Caux Round Table Principles for Business' and the use of online reverse auctions in purchasing.

u 2002 Emiliani / Stec B-2-B institution and

Discusses different kinds of savings resulting from the use of auctions in purchasing. Especially distinguishes between gross and net savings and explores reasons for systematic overestimations of expected savings.

s 2002 Gupta B-2-G process purchasing no

Presents and tests a model to survey the impact of additional bidders on price and auction outcome for road construction auctions of the Florida Department of Transportation.

s 2002 Hong / Shum B-2-G process purchasing no

Tests a model in which bidder's evaluation of the auction item is influenced by both the independent private value model and the common value model based upon 1,018 auctions from the New Jersey Department of Transportation (NJDOT).

s 2002 Jap B-2-B institution and

Based upon a systematic comparison of electronic reverse auctions in contrast to classic auctions, conditions and structures and effects of electronic reverse auctions in purchasing are intensely evaluated.

u 2002 Jap / Mohr B-2-B institution and

Develops a matrix on how exchange types (relational vs. transactional) are related to web-based efficiencies. The excessive use of price-only auctions can indeed jeopardize buyer-supplier-relationships.

s 2002 Koppius / van Heck B-2-B institution n/a no

Empirical analysis whether bidding behavior differs among physically present bidders, online bidders, and online bidders having physical access to the bidding room.

(11)

Source Year of

Abstract

s 2002 Oh B/C-2-C process sales no Compares the likelihood and the extent of a 'Winner's Curse' in C-2-C

and B-2-C auctions.

u 2002 Pearcy / Giunipero /

Dandeo B-2-B institution and

Develops a conceptual model to research the impact of purchase type on the reverse auction process. Furthermore, the impact of auctions on savings, employee's productivity and buyer-supplier-cooperation is analyzed.

s 2002 Smart / Harrison B-2-B process purchasing no

Starts with a brief introduction on current developments in purchasing management and online reverse auctions. Conducts six case studies and researches prices and savings in auctions as well as buyer-supplier- relationships.

s 2002 Smeltzer / Carr B-2-B institution and

process purchasing yes (limited)

Case-based elaboration of the reasons, the pros and the cons of auction use in purchasing and sales. Presents four conditions for an appropriate use of auctions.

s 2002 van Tulder / Mol B-2-B institution and

Starts with a brief overview over the general trade-offs of auction use in terms of strategy, participants and supplier location. Discusses the impact of auctions on prices and buyer-supplier-relations for auctions of cabinets for X-ray machines at Philips Medical Systems.

s 2002 Wurman / Wellman /

Walsh B-2-B/C process n/a no Describes auction rules in terms of bidding, clearing and information for

the use of electronic software agents in auctions.

s 2003 Bajari / McMillan /

Tadelis B-2-B process purchasing no

Analyzes the benefits of auctions compared to face to face negotiations for private construction contracts in California based upon 4,100 awarding procedures held between 1995 and 2000.

u 2003 Beall et al. B-2-B institution and

process

purchasing

and sales yes

Comprehensive and sophisticated analyses of use, design, success factors and future trends for online reverse auctions in purchasing based upon interviews with auction users, non-users, suppliers and auction service providers.

s 2003 Hohner et al. B-2-B process purchasing yes

Considering Mars as example, this articles presents two types of auctions ('combinatorial' and 'volume discount') that meet Mars' requirements for auctions in a complex environment.

u 2003 Jap B-2-B institution and

process

purchasing

and sales no

Researches the impact of auction type (open vs. sealed bid) on buyer- supplier-relationship. Among other insights, auctions can indeed confirm suppliers' concern that buyers may act opportunistically.

u 2003 Kaufmann / Carter B-2-B institution and

Based upon several case studies, this articles highlights that purchasing processes employing electronic reverse auctions are usually not purely electronically, but auction-integrated. Furthermore, conditions and benefits of such an auction-integrated purchasing process in terms of product, market and participants are evaluated.

s 2003 Luedtke B-2-B institution and

Comprehensive work on the use of auctions in purchasing. Discusses influencing factors and design parameters for reverse auctions in purchasing based upon interviews and case studies and concludes with a set of five auction modules that are recommendable for a specific situation.

s 2003 Oliva B-2-B process sales no

Describes four different strategies how suppliers can participate in purchasing auctions and how purchasing managers perceive those strategies. Highlights that suppliers should promote not only the product itself, but also all kinds of options in order to differentiate from competitors.

s 2003 Ottaway / Bruneau /

Evans C-2-C process purchasing no

Researches the impact of available pictures of the auction item, buyers' experiences and sellers' reputation on auction prices (based upon 120 coin auctions at eBay).

u 2003 Pearcy / Giunipero /

Dandeo B-2-B institution purchasing no

Empirically tests the relation between corporate strategy and the use of online reverse auctions. Among other insights, this study does not confirm the relationship between a company pursuing a low cost strategy and the use of 'bid and buy' online reverse auctions.

s 2003 Rothkopf / Harstadt /

Fu B-2-G process purchasing no

Discusses a model about the impacts of subsidizing discriminated bidders (e.g. small or minority-owned companies) in an electronic reverse auction. Focuses on the trade-off between allocation efficiency and bidding dynamics in an auction.

u 2004 Carter et al. B-2-B institution and

Comprehensive analysis of design, use and trends of online reverse auctions in purchasing based upon 46 case studies with auction users, suppliers, and auction service providers.

(12)

Source Year of

Publication Author(s) Sample Size Data Collection Process Auction Item Informants Statistical Analysis s 1986 Luton / McAfee

s 1989 Dasgupto / Spulber s 1992 Anton / Yao

s 1993 Porter / Zona 575 bids

secondary data from the Department of Transportation of Nassau und

Suffolk County

road

construction suppliers multivariate

s 1996 Hallwood

s 1996 Snyder

s 1997 Bag

s 1997 Branco

s 1998 Rodrígez-Aguilar et al.

s 1998 Wolfram > 500 company database (secondary data) electricity National Power &

PowerGen multivariate s 1998 Wolfstetter

s 1999 Beckmann 159 survey n/a

conventional auction service providers and

auctioneers

descriptive and multivariate

s 1999 Sareen 37 database of Indian Oil Corporation

(secondary data) crude oil suppliers

descriptive and multivariate s 1999 Teich / Wallenius /

Wallenius u 2000 Emiliani

s 2000 Jap 6 n/a purchased parts suppliers none

s 2000 Kinney

s 2000 Kjerstad / Vagstad

s 2000 Marshall

s 2000 Wang

s 2000 Wilcox 535 eBay database (secondary data) miscellaneous eBay user

s 2001 Bajari

s 2001 Buchwalter

s 2001 Elgart

u 2001 Emiliani / Stec u 2001 Hong / Hartley s 2001 Kannan / Kopalle

s 2001 Oliva

u 2001 Rosenkranz

s 2001 Sim / Wong

s 2002 Bajari / Summers s 2002 Compte / Jehiel s 2002 Davis / Wilson u 2002 Emiliani / Stec u 2002 Emiliani / Stec

no empirical support

brief reproduction of two existing case studies no empirical support

no empirical support no empirical support no empirical support

Bibliography Empirical Support

no empirical support no empirical support no empirical support

no empirical support no empirical support no empirical support no empirical support no empirical support

no empirical support no empirical support no empirical support no empirical support no empirical support no empirical support

no empirical support no empirical support no empirical support no empirical support

Figure 2: Summary in Terms of Methodology (1/2)

(13)

Source Year of

Publication Author(s) Sample Size Data Collection Process Auction Item Informants Statistical Analysis

s 2002 Gupta 1.937 database of the Florida Department

of Transportation (secondary data)

road

construction suppliers multivariate

s 2002 Hong / Shum 1.018

database of the New Jersey Department of Transportation

(secondary data)

3 types of construction

work

suppliers

s 2002 Jap 38 survey (online) miscellaneous purchasing manager multivariate

u 2002 Jap / Mohr

s 2002 Koppius / van Heck 81.803 database of KOA initiative (secondary

data) flowers purchasing manager multivariate

s 2002 Kwak

s 2002 Mabert / Skeels s 2002 MacDonald / Handy /

Plato 25.000

database of the United States Department of Agriculture (secondary

data)

staple foods suppliers multivariate

s 2002 McDonald / Slawson Jr. 460 eBay database (secondary data)

Harley Davidson collector's barby

doll

eBay user

s 2002 Naegelen

s 2002 Niemeier

s 2002 Oh 600 software-agent ('Price Collecting

Software Agent')

computer

accessories suppliers

descriptive and multivariate u 2002 Pearcy / Giunipero /

Dandeo

s 2002 Smart / Harrison 6 exploratory case studies (secondary

data) miscellaneous purchasing manager none

s 2002 Smeltzer / Carr 41 interview miscellaneous purchasing manager none

s 2002 van Tulder / Mol s 2002 Wurman / Wellman /

Walsh

s 2003 Bajari / McMillan /

Tadelis 4.100 data purchased from Construction Market Data Group (se)

private construction work in northern

California

suppliers

u 2003 Beall et al. 50 interview miscellaneous

purchasing manager, supplier, non-user, auction service provider

descriptive s 2003 Hohner et al.

u 2003 Jap 68 survey miscellaneous suppliers

descriptive and multivariate u 2003 Kaufmann / Carter 73 32 interviews and 41 case studies miscellaneous

purchasing manager, supplier, non-user, auction service provider

descriptive s 2003 Luedtke 35 25 interviews and 10 case studies miscellaneous purchasing manager descriptive

s 2003 Oliva

s 2003 Ottaway / Bruneau /

Evans 120 eBay database (secondary data) coins eBay user

descriptive and multivariate u 2003 Pearcy / Giunipero /

Dandeo 142 survey n/a ISM members

descriptive and multivariate s 2003 Rothkopf / Harstadt / Fu

u 2004 Carter et al. 46 15 interviews und 31 case studies miscellaneous

purchasing manager, supplier, auction service

provider

descriptive

Bibliography Empirical Support

no empirical support case study 'Mars' no empirical support no empirical support no empirical support

case study Fortune 100 company (anonymous)

case study 'Philips Medical Systems' analyses the German UMTS auction

Figure 2: Summary in Terms of Methodology (2/2)

(14)

conceptual empirical (case studies)

empirical (large scale, primary data) B/C-2-C

B-2-G B-2-B

• Luton/McAfee (1986)

• Dasgupto/Spulber (1989)

• Anton/Yao (1992)

• Kjerstad/Vagstad (2000)

• Wang (2000)

• Elgart (2001)

• Bajari/Summers (2002)

• Davis/Wilson (2002)

• Rothkopf/Harstadt/Fu (2003)

• Porter/Zona (1993)

• Sareen (1999)

• Gupta (2002)

• Hong/Shum (2002)

• MacDonald/Handy/Plato (2002)

• Hallwood (1996)

• Teich/Wallenius/ Wallenius (1999)

• Emiliani (2000)

• Kinney (2000)

• Marshall (2000)

• Buchwalter (2001)

• Emiliani (2001, 2002 a,b)

• Hong/Hartley (2001)

• Oliva (2001)

• Rosenkranz (2001)

• Jap/Mohr (2002)

• Kwak (2002)

• Pearcy/Giunipero/Dandeo (2002)

• Wurman/Wellman/Walsh (2002)

• Oliva (2003)

• Kannan/Kopalle (2001)

• Wurman/Wellman/Walsh (2002)

empirical (large scale, secondary data)

• Wolfram (1998)

• Koppius/van Heck (2002)

• Bajari/McMillan/Tadelis (2003)

• Oh (2002)

• Wilcox (2000)

• McDonald/Slawson Jr.

(2002)

• Ottaway/Bruneau/Evans (2003)

• Jap (2000)

• Mabert/Skeels (2002)

• Smart/Harrison (2002)

• Smeltzer/Carr (2002)

• van Tulder/Mol (2002)

• Beall et al. (2003)

• Hohner et al. (2003)

• Kaufmann/Carter (2003)

• Lüdtke (2003)

• Carter et al. (2004)

• Jap (2002 and 2003)

• Pearcy/Giunipero/Dandeo (2003)

no institutional

aspect

• Snyder (1996)

• Bag (1997)

• Branco (1997)

• Rodrigez-Aguilar et al. (1998)

• Wolfstetter (1998)

• Bajari (2001)

• Sim/Wong (2001)

• Compte/Jehiel (2002)

• Naegelen (2002)

• Niemeier (2002)

• Beckmann (1999)

Figure 3: Synopsis in Terms of Content and Methodology

(15)

References:

Amor, D. (2000): Dynamic Commerce, Bonn.

Anton, J. J./Yao, D. A. (1992): Coordination in Split Award Auctions, in: The Quarterly Journal of Economics, Vol. 107, No. 2, pp. 681-707.

Bag, P. K. (1997): Optimal auction design and R&D, in: European Ecomomic Review, Vol. 41, No. 9, pp. 1655-1674.

Bajari, P. (2001): Comparing competition and collusion: a numerical approach, in: Economic Theory, Vol. 18, No. 1, pp. 187-205.

Bajari, P./Summers, G. (2002): Detecting Collusion in Procurement Auctions, in: Antitrust Law Journal, Vol. 70, No. 1, pp. 143-170.

Bajari, P. L./McMillan, R. S./Tadelis, S. (2003): Auctions versus negotiations in procurement: an empirical analysis, Working Paper 9757, National Bureau of Economic Research, Cambridge.

Beall, S. et al. (2003): The Role of Reverse Auctions in Strategic Sourcing (CAPS Research Focus Study), Tempe.

Beckmann, M. (1999): Ökonomische Analyse deutscher Auktionen, Wiesbaden.

Bichler, M. (2001): The Future of e-Markets, Cambridge.

Branco, F. (1997): The design of multidimensional auctions, in: RAND Journal of Economics, Vol. 28, No. 1, pp. 63-81.

Buchwalter, J. (2001): Elektronische Ausschreibungen in der Beschaffung, Cologne.

Carter, C. R. et al. (2004): Reverse auctions – grounded theory from the buyer and supplier perspective, in: Transportation Research Part E [forthcoming].

Compte, O./Jehiel, P. (2002): On the Value of Competition in Procurement Auctions, in:

Econometrica, Vol. 70, No. 1, pp. 343-355.

Cooper, H. (1998): Synthesizing Research: A Guide for Literature Reviews, 3^rd Edition, Thousand Oaks.

(16)

Croom, S./Romano, P./Giannakis, M. (2000): Supply chain management: an analytical frame- work for critical literature review, in: European Journal of Purchasing & Supply Manage- ment, Vol. 6, No. 1, pp. 67-83.

Dasgupto, S./Spulber, D. F. (1989): Managing procurement auctions, in: Information Economics and Policy, Vol. 4, No. 1, pp. 5-29.

Davis, D. D./Wilson, B. J. (2002): Collusion in Procurement Auctions: An Experimental Examination, in: Economic Inquiry, Vol. 40, No. 2, pp. 213-230.

Eisenhardt, K. M. (1989): Making fast strategic decisions in high-velocity environments, in:

Academy of Management Journal, Vol. 32, No. 3, pp. 543-576.

Elgart, E. G. (2001): Army Reverse Auctions: An E-Commerce Acquisition Tool, in: The Public Manager, Vol. 30, No. 1, pp. 13-16.

Ellram, L. (1996): The use of the case study method in logistics research, in: Journal of Business Logistics, Vol. 17, No. 2, pp. 93-138.

Emiliani, M. L. (2000): Business-to-Business online auctions: key issues for purchasing process improvement, in: Supply Chain Management, Vol. 5, No. 4, pp. 176-186.

Emiliani, M. L./Stec, D. F. (2001): Online reverse auction purchasing contracts, in: Supply Chain Management, Vol. 6, No. 3, pp. 101-105.

Emiliani, M. L./Stec, D. F. (2002a): Squaring online reverse auctions with the Caux Round Table Principles for Business, in: Supply Chain Management, Vol. 7, No. 1, pp. 12-23.

Emiliani, M. L./Stec, D. F. (2002b): Realizing Savings from online reverse auctions, in: Supply Chain Management, Vol. 7, No. 1, pp. 92-100.

Engelbrecht-Wiggans, R. (1980): Auctions and Bidding Models – A Survey, in: Management Science, Vol. 26, No. 2, pp. 119-142.

Fink, A. (1998): Conducting research literature reviews: From Paper to the Internet, Thousand Oaks.

Glaser, B. G./Strauss, A. L. (1967): The discovery of grounded theory: Strategies for qualitative research.

Gupta, S. (2002): Competition and Collusion in a Government Procurement Auction Market, in:

Atlantic Economic Journal, Vol. 30, No. 1, pp. 13-25.

(17)

Hallwood, C. P. (1996): A market in action: Fuzzy information and trade-offs between operating rules in the invited tender-bid procurement auction, in: Journal of Economic Studies, Vol.

23, No. 3, pp. 34-43.

Hohner, G. et al. (2003): Combinatorial and Quantity-Discount Procurement Auctions Benefit Mars, Incorporated and Its Suppliers, in: Interfaces, Vol. 33, No. 1, pp. 23-35.

Hong, Y./Hartley, J. L. (2001): An Exploration of On-line Reverse Auctions as a Procurement Tool, in: Proceedings of The 12th Annual North American Research Symposium on Purchasing and Supply Management, Memphis, TN, USA, pp. 79-90.

Hong, H./Shum, M. (2002): Increasing Competition and the Winner's Curse: Evidence from Procurement, in: Review of Economic Studies, Vol. 69, No. 4, pp. 871-898.

Jap, S. D. (2000): Going, Going, Gone, in: Harvard Business Review, Vol. 78, No. 6, p. 30.

Jap, S. D. (2002): Online Reverse Auctions: Issues, Themes, and Prospects for the Future, in:

Journal of The Academy of Marketing Science, Vol. 30, No. 4, pp. 506-525.

Jap, S. D. (2003): An Exploratory Study of the Introduction of Online Reverse Auctions, in:

Journal of Marketing, Vol. 67, July 2003, pp. 96-107.

Jap, S. D./Mohr, J. J. (2002): Leveraging Internet Technologies in B2B Relationships, in:

California Management Review, Vol. 44, No. 4, pp. 24-38

Kannan, P. K./Kopalle, P. K. (2001): Dynamic Pricing on the Internet: Importance and Implications for Consumer Behaviour, in: International Journal of Electronic Commerce, Vol. 5, No. 3, pp. 63-88.

Kaufmann, L./Carter, C. R. (2003): Deciding on the mode of negotiation: to auction or not to auction electronically, in: Proceedings of The 14th Annual North American Research Symposium on Purchasing and Supply Management, Tempe, AZ, USA, pp. 191-226.

Kinney, S. (2000): RIP Fixed Pricing: The Internet Is on Its Way to ‘Marketizing’ Everything, in: Business Economics, Vol. 35, No. 2, pp. 39-44.

Kjerstad, E./Vagstad, S. (2000): Procurement auctions with entry of bidders, in: International Journal of Industrial Organizations, Vol. 18, No. 8, pp. 1243-1257.

Koppius, O. R./van Heck, E. (2002): The Role of Product Quality Information, Market State Information and Transaction Costs in Electronic Auctions, in: Academy of Management Proceedings 2002 [w/o Vol./No./pp.].

(18)

Kwak, M. (2002): Potential Pitfalls of E-Auctions, in: MIT Sloan Management Review, Vol.

43, No. 2, p. 18.

Luedtke, E. C. (2003): Gestaltung von Online-Auktionen im Einkauf, Munich.

Luton, R./McAfee, R. P. (1986): Sequential Procurement Auctions, in: Journal of Public Economics, Vol. 31, No. 2, pp. 181-195.

Mabert, V. A./Skeels, J. A. (2002): Internet reverse auctions: Valuable tool in experienced hands, in: Business Horizons, Vol. 45, No. 4, pp. 70-76.

MacDonald, J. M./Handy, C. R./Plato, G. E. (2002): Competition and Prices in USDA Commodity Procurement, in: Southern Economic Journal, Vol. 69, No. 1, pp. 128-143.

Marshall, J. (2000): Doing Swaps on The Web, in: Financial Executive, Vol. 16, No. 5, p. 8.

McAfee, R. P./McMillan, J. (1987): Auctions and Bidding, in: Journal of Economic Literature, Vol. 25, No. 2, pp. 699-738.

McDonald, C. G./Slawson Jr., V. C. (2002): Reputation in an Internet Auction Market, in:

Economic Inquiry, Vol. 40, No. 3, pp. 633-650.

Milgrom, P. R./Weber, R. J. (1982): A Theory of Auctions and Competitive Bidding, in:

Econometrica, Vol. 50, No. 5, pp. 1089-1122.

Naegelen, F. (2002): Implementing optimal procurement auctions with exogenous quality, in:

Review of Economic Design, Vol. 7, No. 2, pp. 135-153.

Niemeier, S. (2002): Die deutsche UMTS-Auktion, Wiesbaden.

Niessen, H. (1974): Preisbildung auf Sondermärkten, Wiesbaden.

Oh, W. (2002): C2C Versus B2C: A Comparison of the Winner’s Curse in Two Types of Electronic Auctions, in: International Journal of Electronic Commerce, Vol. 6, No. 4, pp.

115-138.

Oliva, R. A. (2001): Discovering Price Discovery, in: Marketing Management, Vol. 10, No. 4, pp. 46-48.

Oliva, R. A. (2003): Sold on Reverse Auctions, in: Marketing Management, Vol. 12, No. 2, pp.

44-46.

(19)

Ottaway, T. A./Bruneau, C. L./Evans, G. E. (2003): The Impact of Auction Item Image and Buyer/Seller Feedback Rating on Electronic Auctions, in: Journal of Computer Information Systems, Vol. 43, No. 3, pp. 56-60.

Pearcy, D. H./Giunipero, L. C./Dandeo, L. M. (2002): The impact of purchase type on the reverse auctions process: a conceptual model , in: Proceedings of The 13th Annual North American Research Symposium on Purchasing and Supply Management, Dallas, TX, USA, pp. 327-346.

Pearcy, D. H./Giunipero, L. C./Dandeo, L. M. (2003): Reverse auction implementation and the impact of corporate strategy on governance structure, in: Proceedings of The 14th Annual North American Research Symposium on Purchasing and Supply Management, Tempe, AZ, USA, pp. 393-407.

Porter, R. H./Zona, J. D. (1993): Detection of Bid Rigging in Procurement Auctions, in: Journal of Political Economy, Vol. 101, No. 3, pp. 518-538.

Rathgen, K. (1911): Auktionen, in: Wörterbuch der Volkswirtschaft, 3rd Edition, No. 1, Jena, pp.

278-279.

Riley, J. G./Samuelson, W. F. (1981): Optimal Auctions, in: The American Economic Review, Vol. 71, No. 3, pp. 381-392.

Rodrígez-Aguilar, J. A. et al. (1998): Towards a test-bed for trading agents in electronic auction markets, in: AI Communications, Vol. 11, No. 1, pp. 5-19.

Rosenkranz, S. (2001): To Reveal or Not to Reveal: Know-How Disclosure and Joint Ventures in Procurement Auctions, in: Journal of Institutional and Theoretical Economics, Vol.

157, No. 4, pp. 555-567.

Rothkopf, M. H./Harstadt, R. M./Fu, Y. (2003): Is Subsidizing Inefficient Bidders Actually Costly?, in: Management Science, Vol. 49, No. 1, pp. 71-84.

Sareen, S. (1999): Posterior Odds Comparison of a Symmetric Low-Price, Sealed Bid Auction within the Common-Value and the Independent-Private-Values Paradigms, in: Journal of Applied Econometrics, Vol. 14, No. 6, pp. 651-676.

Schlinghoff, A./Backes-Gellner, U. (2002): Publikationsindikatoren und die Stabilität wirtschaftswissenschaftlicher Zeitschriftenrankings, in: Schmalenbachs Zeitschrift für betriebswirtschaftliche Forschung (zfbf), Vol. 54, June 2002, pp. 343-362.

Sim, K. M./Wong, E. (2001): Toward Market-Driven Agents for Electronic Auction, in: IEEE Transactions on Systems, Man & Cybernetics – Part A: Systems and Humans, Vol. 31, No. 6, pp. 474-484.