1
Dissecting the Canon:
Visual Subject Co-Popularity Networks in Art Research
Maximilian Schich1 with Sune Lehmann2 and Juyong Park2
1. Bibliotheca Hertziana (Max-Planck-Institute for Art History), Rome, Italy (www.biblhertz.it) 2. Center for Complex Network Research, Northeastern University, Boston/MA, USA (www.barabasilab.com)
Introduction
In Art History and Archaeology scholars use documents to study ob- jects together with their meaning, rela- ted people, locations, times and events.
Within this effort Art History has been defined as the history of all man-made things (Kubler, 1962), which implies a focus on the dynamics of interrelated objects – the growth of what can be seen as the coral reef of culture (Gombrich, 1979).
An important question in this domain is the definition or emergence of ca- non, i.e. the set of most popular objects, which everybody knows or supposedly
should know in a given area – such as Da Vinci’s Mona Lisa and Botticelli‘s Venus in painting or the Colosseum and the Pantheon in architecture.
In this paper we show that canons are identical with the most popular items over a distribution of popularity, which happens to be highly heterogeneous. As a consequence we can explore the me- aning of canon by looking at the co-po- pularity of visual objects in general, no matter if the objects belong to the head or the tail of the popularity distributi- on.
This paper was presented in a long oral presentation at ECCS2008, Jerusalem, Israel. At the International Workshop on Challenges and Visions in the Social Sciences, ETH Zurich, Switzerland (August 18-23, 2008) the respective poster received
a Best Poster Award. The paper was first published in the ECCS online conference material on September 3, 2008.
(see http://www.jeruccs2008.org/node/114)
AR T-Dok postprint URN:nbn:de:bsz:16-artdok-7111
2
Background
New research in the area of co-popu- larity has been facilitated recently by the emergence of relevant datasets, in which tags and other classifications have been used to classify a large number of images and image segments. The work in these projects is either done manually by hu- man editors (Schich 2007, Russell 2008), automatically with the help of pattern recognition algorithms (e.g. http://www.
definiens.com) or by human computa- tion, i.e. in a collaborative effort in the form of games such as Peekaboom (Ahn 2006).
The data produced by these efforts can be understood as bi-partite networks connecting image documents and classi- fication criteria. Moreover image docu- ments as well as the classification criteria may feature additional information in the form of trees or ontologies (cf. figure 1).
As shown in at least two studies (Schich 2007, Russell 2008), such bi-partite clas- sification networks usually belong to the class of scale-free networks, characte- rized by a highly heterogeneous connec- tivity distribution (cf. figure 2). Hence
methods developed in network science can be used to process art research data in search for better definitions of a canon.
Subject Popularity
In this paper we analyze a classic da- taset of art research, which collects anci- ent art and architecture and their Western Renaissance documentation since 1947 (CENSUS 2005):
As we can see in the plot in figure 2, there is clearly a long tail of monument popularity, no matter if we look at the Number of Renaissance Documents •, the Number of Depictions/Descriptions in the Documents or the Total Num- ber of Links Including Overpopulation
(where single depictions are linked to multiple monument parts).
In addition the long tail emerging from the Number of Documents can be dissected into tails of different character, such as Non-Architectural Sculpture + and Everything Else x.
The hitlist in figure 3 gives a clear idea how Non-Architectural Sculpture, Ar- chitecture and Sculptural Architecture combine to the general canon of ancient monuments in Western Renaissance.
classification criterion including subdivisions (for e.g. a monument)
document or object including subdivisions
classification/document edge including n links between (sub-)nodes
images of document (sub-)nodes
painting section 1
section 3 section 2 blob
bowl fish
blob
bowl fish
painting 1 painting 2
fig. 1 A simple example of a bi-partite image classification network, where paintings and their classified seg- ments are represented as trees, which are connected to classification criteria via the classification link.
3
0,001 0,01 0,1 1
1 10 100 1000 10000
0,001 0,01 0,1 1
1 10 100 1000
IN-degree of Ancient Monuments Disections of IN-degree(Number of Renaissance Documents) dark blue: Number of linked Renaissance Documents dark blue: Ancient Monuments in general
blue: Occurence in Renaissance Documents blue +: Non-architectural sculpture
(i.e. number of depictions/descriptions) light blue: Statues of Venus/Aphrodite
light blue: Total number of links (including overpopulation) orange x: All except non-architectural sculpture
0,0001 0,0001
0,001 0,01 0,1 1
1 10 100 1000 10000
0,001 0,01 0,1 1
1 10 100 1000
IN-degree of Ancient Monuments Disections of IN-degree(Number of Renaissance Documents) dark blue: Number of linked Renaissance Documents dark blue: Ancient Monuments in general
blue: Occurence in Renaissance Documents blue +: Non-architectural sculpture
(i.e. number of depictions/descriptions) light blue: Statues of Venus/Aphrodite
light blue: Total number of links (including overpopulation) orange x: All except non-architectural sculpture
0,0001 0,0001
NodeID Ancient Monument Renaissance documents occurence in documents total links link overpopulation
NodeID Ancient Monument (non-architectural sculpture) Ren
aissance documents occurence in documents total links link overpopulation
NodeID Ancient Monument (statues of Venus/Aphrodite) Ren
aissance documents occurence in documents total links link overpopulation
150908 Arch of Constantine (triumphal arch) 144 360 764 112% 219823Laocoon (group of statues) 98 156 156 0% 156661 Female Figure (statue) 15 17 17 0%
150770 Pantheon (temple) 134 629 1131 80% 150776Horsetamers (group of statues) 75 107 108 1% 156022 Venus Felix (group of statues) 15 16 16 0%
150940 Arch of Septimius Severus (triumphal arch) 114 329 754 129% 151697Equestrian Statue of Marcus Aurelius (equestrian statue) 61 94 94 0% 154662 Venus Belvedere (statue) 12 15 15 0%
150792 Colosseum (amphitheatre) 100 457 642 40% 150779Apollo Belvedere (statue) 58 66 66 0% 158421 Venus Genetrix (statue) 9 9 9 0%
219823 Laocoon (group of statues) 98 156 156 0% 234323Regisole (equestrian statue) 49 80 80 0% 156203 Venus ex Balneo (statue) 8 8 8 0%
151057 Column of Trajan (honorific column) 90 261 363 39% 151625Bacchic Sarcophagus (sarcophagus) 45 75 75 0% 151527 Crouching Venus (statue) 7 12 12 0%
150958 Arch of Titus (triumphal arch) 89 264 372 41% 151526Torso Belvedere (statue) 42 53 53 0% 159346 Venus (statue) 7 7 7 0%
150812 Baths of Diocletian (thermae) 80 314 506 61% 153508 Vitellius (bust) 40 40 40 0% 156208 Crouching Venus with Tortoise (statue) 6 7 7 0%
150826 Basilica of Constantine (basilica) 78 198 268 35% 155031 Hercules (statue) 36 51 51 0% 157758 Venus (statue) 6 7 7 0%
150784 Temple of Mars Ultor (temple) 75 159 338 113% 155719 Spinario (statue) 35 36 36 0% 151588 Diana of Ephesos (statue) 5 7 7 0%
150776 Horsetamers (group of statues) 75 107 108 1% 155402 Lupa Capitolina (statue) 33 34 34 0% 157183 Venus Anadyomene (statue) 5 6 6 0%
151227 Forum of Nerva (forum) 74 172 273 59% 151330 Rivergod Marforio (statue) 32 37 37 0% 161469 Nude Female Torso (statue) 5 6 6 0%
150844 Baths of Caracalla (thermae) 70 275 506 84% 155401 Horses of San Marco (group of statues) 29 31 31 0% 156882 Goddess (statue) 5 5 5 0%
150890 Theatre of Marcellus (theatre) 70 205 366 79% 151737 Nile (statue) 27 32 32 0% 158343 Venus Medici (statue) 4 8 8 0%
151328 Temple of Antoninus and Faustina (temple) 62 160 228 43% 151738 Tigris (statue) 26 30 30 0% 156207 Crouching Venus (statue) 4 6 6 0%
151697 Equestrian Statue of Marcus Aurelius (equestrian statue) 61 94 94 0% 152035 Apollo (statue) 25 32 46 44% 156732 Venus (statue) 4 5 5 0%
150779 Apollo Belvedere (statue) 58 66 66 0% 151520 Nile (group of statues) 24 41 41 0% 156684 Venus (statue) 4 4 4 0%
151259 Mausoleum of Hadrian (sepulchral monument) 57 125 142 14% 151521 River God Tiber (group of statues) 24 28 28 0% 157516 Ceres (statue) 4 4 4 0%
151930 Temple of Minerva (temple) 56 149 212 42% 153877 Apollo Farnese (statue) 24 25 25 0% 159373 Venus (statue) 4 4 4 0%
150806 Septizonium (facade) 56 118 124 5% 155419 Trophy with Cuirass (statue group) 22 26 26 0% 159632 Venus (statue) 4 4 4 0%
151038 Temple of Castor and Pollux (temple) 55 153 207 35% 151514 Antinous Belvedere (statue) 21 22 22 0% 156204 Venus (statue) 3 8 8 0%
234323 Regisole (equestrian statue) 49 80 80 0% 156663 Hercules Farnese (statue) 21 22 22 0% 157182 Venus Binding her Sandal (statue) 3 8 8 0%
151320 Temple of Saturn (temple) 46 110 145 32% 151507 Hercules and Telephos (group of statues) 20 23 23 0% 161753 Venus (statue) 3 5 5 0%
151322 Curia Julia (curia) 45 95 112 18% 158989 Funerary Monument (relief) 20 23 23 0% 155784 Venus Santa Croce (statue) 3 4 4 0%
151625 Bacchic Sarcophagus (sarcophagus) 45 75 75 0% 152103 Ariadne (statue) 20 22 22 0% 156750 Leda and the Swan (statue) 3 4 4 0%
151065 Temple of Serapis (temple) 44 120 175 46% 155687 Piping Marsyas (statuette) 20 20 20 0% 157973 Venus (statue) 3 4 4 0%
150785 Forum Augustum (forum) 43 90 129 43% 155420 Trophy with Fur Mantle (statue group) 19 23 23 0% 161783 Torso of Venus (statue) 3 4 4 0%
151046 Forum of Trajan (forum) 42 82 90 10% 153320 Funerary Altar of T. Julius Aug. Mnester (funerary altar) 19 20 20 0% 157560 Venus (statue) 3 3 3 0%
151526 Torso Belvedere (statue) 42 53 53 0% 151533 Capitoline Pan (statue) 19 19 19 0% 158982 Venus (statue) 3 3 3 0%
151143 Basilica Aemilia (basilica) 41 117 176 50% 156185 Seated Nymph (statue) 19 19 19 0% 159376 Sea Goddess (statue) 3 3 3 0%
fig. 3 The Top 30 hitlist of monument popularity, defined by the number of Renaissance documents, clearly corresponds to the expected canon of ancient monuments in Western Renaissance.
fig. 2 Cumulative distibutions of various types of monument in-degree in the CENSUS 2005 dataset. The plot indicates the probability Pk (y-axis) that a monument node has at least a certain number of connections k (x-axis). See text for the various types of connections.
P
kCanons are tails within tails!
Extrapolating from the result that the long tail of ancient monument po- pularity in Western Renaissance can be dissected into various sub-tails, the ge- neral canon of art history can be seen as the head of the long tail distribution of object popularity, where the sub-canon of given specialized areas appears as the head of a self-similar sub-tail of the whole distribution.
In the examples in figures 4, 5, and 6 we size object images according to their documentation frequency, which provi- des us with a limiting condition of what objects are contained in various canons emerging from the documents:
The first example in figures 4 and 5 shows the long tail of Non-Architec- tural Sculpture in analogy to the + plot and the blue entries in the hitlist in figu- res 2 and 3.
The second example in figure 6 pre- sents the top 30 monuments of the sub- tail of Statues Identified as Venus or Aphrodite at some point in history (ac- cording to the Census database). Again the long tail appears in the • plot in fi- gure 2.
Note: For each monument in figure 4, 5 and 6 we show a directly attached photo or an image of the first document. Question marks indicate that the mo- nument is untraced, i.e. lost since the Renaissance and only verbally documented, or without image infor- mation at the first linked document in the database.
fig. 4 The long tail of Non-Architectural Sculpture.
5
fig. 6 The sub-tail of Statues Identified as Venus or Aphrodite has very similar properties as the general long tail of popularity. The same is true for any other chosen sub-tail. There is no average popularity for any class of monuments. Instead, we find long tails of sarcophagi, column bases, temples or any other category.
fig. 5 The head of the long tail of Non-Architectural Sculpture is identical to the respective sub-canon as any specialist would expect it. However, the canon is also clarified: in addition to good old friends like the Laoko- on, the Horsetamers, the Marc-Aurelius equestrian monument, the Apollo Belvedere, and so on, there are also some surprises, such as five river gods within the top 18.
Note that this canon is not defined by some central authority, but emerges from the documents, whose monu- ment selection varies highly both in genre as well as in number.
classification criterion including subdivisions (for e.g. a monument)
document or object including subdivisions
classification/document edge including n links
between (sub-)nodes images of document (sub-)nodes
painting
section 1
section 3 section 2 blob
bowl fish
blob
bowl fish
painting 1 painting 2 painting 3
classification criterion including subdivisions (for e.g. a monument)
document or object including subdivisions
classification/document edge including n links
between (sub-)nodes images of document (sub-)nodes
painting section 1
section 3 section 2 blob
bowl fish
blob
bowl fish
painting 1 painting 2 painting 3
Visual Subject Co-Popularity
Extending from the question of po- pularity and canon, we present a new way to explore the related phenomenon of visual subject co-popularity. Starting from a classified/annotated image data- set, we propose a method which com- bines a bi-partite community-finding algorithm and a method for the produc- tion of scalable image matrices in order to construct 2-dimensional overviews.
In order to find interesting areas in the whole network we apply a commu- nity-detection algorithm for overlap- ping bi-cliques introduced by Lehmann et al. (2008), which generalizes on the k-clique community finding algorithm for one-mode networks by Palla et al.
(2005).
In a second step the communities found by the algorithm are visualized using a method for the production of
scalable image matrices introduced by Schich (2008). Here, node information of a bi-partite classification network is placed in the location of the links in the adjacency matrix of the network, as shown in figure 7 for the simple pain- tings example (cf. figure 1) and the monument-document network in the CENSUS dataset.
The figures 8a-c provide a proof of concept for our method. The resulting image matrix obviously indicates some reasons of co-popularity of otherwise unrelated monuments - in our case all monuments except for the two super- prominent river gods were obviously located in topographical proximity in the mid 16th century.
Note how even this small selection of monuments forms another sub-tail of popularity indicated by the red frames in the figures 4, 5 and 6.
fig. 7 In order to produce 2-dimensional overviews, node information of the image partition is placed in the location of the links in the adjacency matrix (cf. our paintings to the left). While simple in principle, this method can be complicated by the complexity of the node information (cf. CENSUS to the right).
7 c
s u n e V s
u n e V g n i h c u o r C e
li N e
s a V h t i w y o B n
o t i e g o t s i r A o
i r o f r a M d o g r e v i R
) e u t a t s ( )
e u t a t s ( )
s e u t a t s f o p u o r g ( )
e u t a t s ( )
e u t a t s ( )
e u t a t s (
8 5 7 7 5 1 7
2 5 1 5 1 0
2 5 1 5 1 9
0 5 1 5 1 4
0 5 1 5 1 0
3 3 1 5 1
6 0 7 1 6 3
9 2 0 6 4
4 1 0
6 61203 61296 60185
: o m a l o r i G , i p r a C a D : d e t u b i r t t a n u : s i s n e g i r b a t n a C s u m y n o n
A Aldroandi, Ulisse: Aldroandi, Ulisse: unattributed:
Cambridge, Trinity Coll. Berlin, SMBPK, Kupferstichkabinett Torino, Bibl. Reale Aldroandi1562 Aldroandi1556 Berlin, SMBPK, Kupferstichkabinett
Cambridge Sketchbook Heemskerck Album II Contraffazioni (1562) (1556) Heemskerck Skb. I
) 3 5 5 1 - 9 4 5 1 ( )
d e t a d n u ( ) 2 6 5 1 e t n a - 0 5 5 1 t s o p
( [textonly] [textonly] (undated)
Rivergod Marforio (statue) Aristogeiton (statue) Boy with Vase (statue) Nile (group of statues) Crouching Venus (statue) Venus (statue) Monuments
151330 151504 151509 151520 151527 157758
RecNo Documents
x x x x 60144 Anonymus Cantabrigensis: Cambridge, Trinity Coll. / Cambridge Sketchbook / (post 1550-ante 1562) x x x x x 60293 unattributed: Berlin, SMBPK, Kupferstichkabinett / Heemskerck Album II / (undated)
x x x x x 61706 Da Carpi, Girolamo: Torino, Bibl. Reale / Contraffazioni / (1549-1553) x x x x x x 61203 Aldroandi, Ulisse: Aldroandi 1562 / (1562) [text only]
x x x x x x 61296 Aldroandi, Ulisse: Aldroandi 1556 / (1556) [text only]
x x x x x x 60185 unattributed: Berlin, SMBPK, Kupferstichkabinett / Heemskerck Skb. I / (undated) b a
s u n e V s
u n e V g n i h c u o r C e
li N e
s a V h t i w y o B n
o t i e g o t s i r A o
i r o f r a M d o g r e v i R
) e u t a t s ( )
e u t a t s ( )
s e u t a t s f o p u o r g ( )
e u t a t s ( )
e u t a t s ( )
e u t a t s (
8 5 7 7 5 1 7
2 5 1 5 1 0
2 5 1 5 1 9
0 5 1 5 1 4
0 5 1 5 1 0
3 3 1 5 1
6 0 7 1 6 3
9 2 0 6 4
4 1 0
6 61203 61296 60185
: o m a l o r i G , i p r a C a D : d e t u b i r t t a n u : s i s n e g i r b a t n a C s u m y n o n
A Aldroandi, Ulisse: Aldroandi, Ulisse: unattributed:
Cambridge, Trinity Coll. Berlin, SMBPK, Kupferstichkabinett Torino, Bibl. Reale Aldroandi1562 Aldroandi1556 Berlin, SMBPK, Kupferstichkabinett
Cambridge Sketchbook Heemskerck Album II Contraffazioni (1562) (1556) Heemskerck Skb. I
) 3 5 5 1 - 9 4 5 1 ( )
d e t a d n u ( ) 2 6 5 1 e t n a - 0 5 5 1 t s o p
( [textonly] [textonly] (undated)
Rivergod Marforio (statue) Aristogeiton (statue) Boy with Vase (statue) Nile (group of statues) Crouching Venus (statue) Venus (statue) Monuments
151330 151504 151509 151520 151527 157758
RecNo Documents
x x x x 60144 Anonymus Cantabrigensis: Cambridge, Trinity Coll. / Cambridge Sketchbook / (post 1550-ante 1562) x x x x x 60293 unattributed: Berlin, SMBPK, Kupferstichkabinett / Heemskerck Album II / (undated)
x x x x x 61706 Da Carpi, Girolamo: Torino, Bibl. Reale / Contraffazioni / (1549-1553) x x x x x x 61203 Aldroandi, Ulisse: Aldroandi 1562 / (1562) [text only]
x x x x x x 61296 Aldroandi, Ulisse: Aldroandi 1556 / (1556) [text only]
x x x x x x 60185 unattributed: Berlin, SMBPK, Kupferstichkabinett / Heemskerck Skb. I / (undated)
fig. 8 We explore co-popularity in three steps: First, a community of monuments and documents found by the bi-clique community finder (a) is visualized much more clearly as an adjacency matrix (b). After permu- tation and filtering, the images of subordinate document nodes are finally placed in the location of the links (c). The method scales well to much larger communities.
Discussion
Our approach generalizes the ques- tion of canon in art research using the concept of co-popularity, which also ap- plies to the not so well known part of the long tail.
By introducing the network paradigm in art research we open the door for nu- merous applications on a wide range of art- and archaeology-related datasets.
Besides shedding light on the structure of the canon of art, the resulting image matrices can also be used to investigate a canon’s dynamics, facilitating the recon- struction of the mostly implicit network of visual citation.
In addition our approach has the potential to augment the usual one- dimensional results of image databases and search engines by placing the found image information in a two-dimensional overview, which enables the comparison of multiple classification criteria in mul- tiple images within the context of the network structure.
By using a bi-clique community-fin- ding algorithm our method overcomes the problem of picking the right area in the network, containing a large amount of information while still being useful to the human eye. The approach discovers hidden relationships in the data in a re- producible manner, which otherwise can only be deduced by individual cognitive efforts and which up until now could not be visualized in an objective form.
Future work
The current results are a starting point to explore further issues, such as the superconnected core of co-populari- ty which seems to be a common feature of the investigated classification net- works in art research. We will approach this issue by combining algorithms loo- king for dense communities, like the one used in the present paper with other al- gorithms breaking the core into pieces in order to allow for targeted bottom up recombination of fragments.
Another issue is the ambivalent na- ture of superordinate document and classification entities, which the scalable image matrix method deals with, but for which community finding algorithms have to be adapted.
Finally we plan to investigate not only the structure but also the dynamics of the canons of art history, which in- cludes dealing with the phenomenon of novelty in addition to (co-)popularity (cf. Wu 2008).
9 References
G. Kubler (1962): The Shapes of Time. Remarks on the History of Things. New Haven/London: Yale University Press.
E.H. Gombrich (1979): The Sense of Order. A Study in the Psychology of Decorative Art. Oxford:
Phaidon, pp. 209-210.
G. Palla, I. Derényi, I. Farkas, and T. Vicsek (2005): Uncovering the overlapping community struc- ture of complex networks in nature and society. Na- ture, 435:814, 2005.
CENSUS (2005): Census of Antique Works of Art and Architecture Known in the Renaissance. ed. A.
Nesselrath, Verlag Biering & Brinkmann / Stiftung Archäologie, Munich 1997-2005. http://www.dyabo- la.de
CENSUS (2006...): Census of Antique Works of Art and Architecture Known in the Renaissance. Ber- lin-Brandenburgische Akademie der Wissenschaften and Humboldt-Universität zu Berlin. http://www.
census.de
L. von Ahn, R. Liu, M. Blum (2006): Peekaboom:
A Game for Locating Objects in Images. CHI 2006 Proceedings. April 22-27, 2006, Montréal, Québec, Canada. http://www.peekaboom.org
M. Schich (2007): Rezeption und Tradierung als Komplexes Netzwerk. Der CENSUS und visuelle Do- kumente zu den Thermen in Rom. (Diss.) Humboldt- Universität zu Berlin 2007.
B.C. Russell, A. Torralba, K.P. Murphy, W.T.
Freeman (2008): LabelMe: a database and web-based tool for image annotation. To appear in the Internati- onal Journal of Computer Vision. Revised January 2, 2008. http://labelme.csail.mit.edu/
M. Schich (2008): Method for producing scalable image matrices. PCT/EP2007/006900 WO/2008/017430 http://www.wipo.int/pctdb/en/
wo.jsp?WO=2008017430.
S. Lehmann, M. Schwartz and L.K. Hansen (2008):
Biclique communities. Phys. Rev. E 78, 016108 F. Wu and B.A. Huberman (2008): Populari- ty, novelty and attention. in: Proceedings of the 9th ACM Conference on Electronic Commerce (Chi- cago, Il, USA, July 08-12, 2008). EC ‚08. ACM, New York, NY, 240-245. DOI= http://doi.acm.
org/10.1145/1386790.1386828
Acknowledgements
The method producing scalable image matrices has been filed as a patent with Bibliotheca Hertziana (Max- Planck-Institute for Art History) in Rome. The present work is part of the method‘s further development with the help of an extraordinary fund of the Max-Planck board. The current results on the canon of sculpture in the Renaissance are valuable in the ongoing project with Prof. Ebert-Schifferer and her team at Bibliotheca Hertziana dealing with Repoussoir figures in paintings by Annibale Carracci and Caravaggio. We thank Stiftung Archäologie in Munich for providing us with the CENSUS 2005 data. Our regards also go to Prof. Arnold Nesselrath and the CENSUS team for continuing over 60 years of tradition with the BBAW CENSUS (2006...). Finally special thanks go to Prof. Albert-László Barabási whose inspiration Linked everything together.
