Directed searching vs. Data-driven research


The Art of Counting project is based on the combination of a custom-built relational database and advanced statistical methods.  The database revolves around a core of variables that are recorded in a binary (yes or no) manner.  In the case of my dissertation project, which focused on the king’s regalia, the Medinet Habu database tracked 132 binary variables.  Many of these were focused on pharaoh and his costume–is he wearing a khepresh (Y/N)?, an atef (Y/N)?, horns on headgear (Y/N)?, beard (Y/N)?, sandals (Y/N)?, wine jar in hand (Y/N)?, bow in hand (Y/N)?, disc hovering above (Y/N)?, sunshade behind (Y/N)?, in contact with a deity? etc.  The other actors in the scene (humans?, aggressive foes? Amun? an ithyphallic god?), scene type identifiers (i.e. slaughter?, festival?, offering?, smiting?), and locational data (north wall? portal? column? exterior?) rounds out the layout used to capture Medinet Habu.

By entering data in this way, two things are greatly facilitated–performing directed searches of the database and the application of a wide variety of statistical analyses.  ‘Directed searches’ simply means that the researcher directs what is being investigated; for example, once you have all the data entered, you might choose to see how many times a particular variable occurred (a frequency analysis). Direct searches can be extremely useful and revealing when the researcher has a specific line of questioning in mind.  For instance, if your research is focused on a particular costume variable, such as sandals, using the database you can quickly identify the appearances of any combination of other variables with sandals.  The database then allows you to revise that search in myriad ways to rapidly investigate numerous lines of inquiry.

Data-driven research, on the other hand, can instantly identify all of the patterns apparent in the dataset.  With the data-driven approach, the researcher may choose the analytical method (such as tetrachoric correlation), but will then let the methodology and the data lead the way from that point on. The researcher can then choose the patten that interests them the most and proceed along that path, using directed searches in the database to help flesh out the subtleties of the interactions.

The Art of Counting believes first and foremost that you should let the unfiltered data lead the way and then follow up with directed searches.  Other types of analyses where you apply your ideas to the data (such as cluster analysis where you choose the number of clusters, making it a hybrid of a directed search and a data-driven approach) are quite valuable, but allowing the data to lead the way removes bias and provides discoveries that can’t be found any other way.  Letting the data lead the way is the closest thing we have to having direct access to the ancient Egyptian mindset.

If you’d like to see a video that goes into some more detail about this process, please see here.

Share Button
  1. Lili04-25-2011

    Amy, this is an excellent article. It describes the essence of your research. The idea of data-driven is what makes this research ground breaking. For someone who has no knowledge about ancient Egypt, data-driven research provides a fresh look at the history, or the history we have been “assuming” for years. One of my favorite quote says “the truth is rarely pure and never simple.” What made us so sure about what really happened thousands of years ago? What made us believe that the pictures really told the truth? For years, what we know about ancient Egypt came from visual analysis of numerous researchers. Each one of them introduces his/her own bias into the studies. The time for the data to speak for itself has finally come.

  2. Alex01-29-2012

    I think your project is fascinating and valuable, but I do want to point out that you’ve already filtered the data by creating an iconographic database, so describing the data as unfiltered (and going further to suggest that it speaks for itself) is problematic. It’s possible that the most interesting aspects of the reliefs you’re analyzing may be hidden in the relative depths of the carvings, from which one might extrapolate the number of craftsmen or schools involved in their production, or in any number of other features that you have decided not to include in your database. The correlations that you perceive are effectively pre-clustered, and any search you do of the database is directed by the parameters of the database itself, even if you don’t impose arbitrary groups on the data. This is merely to say that positivist technocratic rhetoric risks alienating people who study material that you have deemed unnecessary to quantify for the purposes of your study, and foreclosing avenues of investigation that depend on variables that you have omitted from your data collection and analysis.

    • Art of Counting02-04-2012

      Thank you for your interest in this project and your insightful comments!
      You are absolutely right, Alex. For the initial foray using a quantifiable and statistical methodology in ancient Egyptian iconographic study, I necessarily focused my attentions. Even if I wanted to go on forever (which could have easily happened), the IFA has a strict text page limit on PhD dissertations. My primary dissertation premise revolved around the evolution of New Kingdom royal regalia, so the majority of the variables tracked in the PhD version of the database were focused on the king’s costume and his context. A great many scene elements beyond pharaoh were recorded; however, it was deemed appropriate for a baseline effort that we keep the number of tracked variables relatively small (around 130). So, you are correct in that sense–my data is filtered and the correlations are pre-clustered because I selected the appropriate variables to study for my particular project.
      Now that the process has clearly displayed huge benefits for the art historical researcher, I have been busily assembling an international team of scholars in an effort to design a master variable list and database capable of capturing ALL the variables we can conceive of, including those interesting physical characteristics that may reveal information about the artists themselves. It certainly wasn’t my intention to alienate anyone–we just haven’t gotten to everything yet!
      Want to join us?

  3. gerd moe-behrens09-15-2012

    Thanks for citing my guest blog. You might also like my recent blog in Nature, SoapBox Science, ToolTales Leukippos – Synthetic Biology Lab in the Cloud
    Moreover I put a talk about this subject online: http://

The Art of Counting is dedicated to the memory of Margery Meilleur, who first taught me to view history through the eyes of the images we create.