How the Oracle of Bacon Works
Every couple of weeks the Oracle retrieves updates from TMDB. We produce a SQL database with 878,519 films, 108,733 TV shows, and 2,530,751 actors and actresses.There is a database service running at all times that stores the database file in memory. It is written in Rust for safety and maximum CPU and memory efficiency. The service handles three different types of requests:
- Find the link from Actor A to Actor B.
- How good a "center" is a given actor?
- Who are all the people with an Actor A number of N?
There are several PHP pages -- one for each of the above types of queries -- that run on the Oracle of Bacon web server, which all connect to the database service using TCP.
The database service uses a breadth-first search (BFS) to find the shortest path between pairs of actors. If you want to dig further into how shortest-path algorithms work, I recommend the textbook by Cormen, Leiserson, Rivest, and Stein as an excellent place to start. Other algorithms textbooks are likely to cover the subject as well, if Introduction to Algorithms isn't available. You may also look at materials that I wrote to explain graph algorithms (including BFS) to Duke undergraduate CS students here.
Whenever the Oracle answers a query, the results are cached so that future requests to link to the same actor will occur more quickly. About 95% of all queries can be served instantly from the result cache. The current contents of the cache (i.e., which actors can be linked quickly) can be found here.
The database server runs on an Amazon EC2 node with Ubuntu Linux. It consumes 500MB of RAM, about half of which is used for the results cache.