List of ODATE Notebooks

These links will launch the notebooks using Binder. It can sometimes take a few moments for the environment to launch; be patient. Click on the ‘status’ link when launching to see the environment build.


Introduction to Jupyter Notebooks

Binder Repository

Contains:

  • Welcome.ipynb
  • demo-R.ipynb

This notebook contains everything necessary to set up a Github repo that can become the basis of a Binder. Consult the repository’s Readme file to see how it can be customized for your own particular usage. Fork (make a copy) of this repo as often as is necessary! Many of the exercises in the first part of ODATE require nothing more than this.


Working with APIs

Binder Repository

Contains:

  • chronicling america api.ipynb
  • open context api.ipynb
  • Open Context Measurements.ipynb
  • mapping-with-ipyleaflet.ipynb

These notebooks demonstrate progressively more complicated ways of retrieving data via an API.


Archaeological Data into R

Binder Repository

Contains:

  • Retrieving Data from the Portable Antiquities Scheme Database.ipynb
  • archdata.ipynb

The first notebook shows how to use R to pull archaeological data from an online database. The second notebook shows how to interact with the archdata package, a collection of archaeological datasets already pulled together for usage in R.


Databases

Binder Repository

Contains:

  • intro to sql.ipynb
  • SQLite Database and R.ipynb
  • visualizing results of sql query in python.ipynb

This notebook demonstrates how to ingest a variety of csv (or other format) files into a single SQL database. It shows how to query the database, and to push the results of the query into a dataframe for further analysis or visualization.


Linked Open Data

Binder Repository

Contains:

  • sparql-intro.ipynb
  • Using R to Retrieve and Visualize Data from SPARQL.ipynb

The first notebook shows how to craft sparql queries for the British Museum, Wikidata, and Nomisma endpoints. The first notebook uses a SPARQL kernel that also allows for graphing visually the data relationships; it also has a ‘magic’ command for writing the data to json or csv. The second notebook demonstrates how to use the sparql package for R to query an endpoint and then manipulate the results to do some simple statistics and visualization.


Spatial Archaeology

Binder Repository

  • linlithgow_spatial.ipynb
  • canmore_survey_shetland.ipynb
  • 1_spatialarchaeology.ipynb
  • working with remote sensing data.ipynb

These notebooks are courtesy Dr. Rachel Opitz, of the University of Glasgow who is Lecturer in Spatial Archaeology. There are two binders below which can be launched; consider launching Dr. Opitz’s as she updates the work for her own teaching, or use the ODATE version that is updated only periodically. Dr. Opitz’s version can be launched here: Binder

The Linlithgow notebook explores burial data, while the Canmore notebook explores the map of registered monuments in the Shetlands, recorded in Scotland’s Canmore database. The 1_spatialarchaeology.ipynb notebook explore’s data from Dr. Opitz’s team’s excavations at the ancient city of Gabii in Italy. The final notebook works with remote sensing data and explores cropmarks in hyperspectral images.


Scraping

Binder Repository

  • Extracting Data from PDFs using Tabulizer.R
  • metadigitise.R
  • Building a Scrapy Scraper.ipynb

To launch the two .R scripts, use the built-in RStudio server in this binder. This binder will take a bit of time to load up.

From the Home page for this binder, select new -> RStudio. Then open the Extracting Data from PDFs using Tabulizer.R file.

Put your cursor at the first line in the script (top left window); run one line at a time.


LiDAR

Binder Repository

  • Demo using Montreal LiDAR data.ipynb
  • Avebury LiDAR.ipynb

These two notebooks show how to unzip .laz files into .las, and to visualize the data therein.

The final codeblock in both notebooks creates an animated gif from the data. That final codeblock is computationally intensive; it will take some time to run. The results will be written to a new folder called ‘export’; you can open that folder by clicking on the jupyter logo at top and then clicking on the ‘export’ folder.

You will know the code is finished when the [*] at the left of the code block changes to a number.

Start with the Montreal demo notebook. It contains some code that the Avebury notebook depends on.


Agent Based Modeling

Binder Repository

Contains:

  • Schelling Segregation Model - schelling/analysis.ipynb
  • Epstein Civil Violence Model - epstein/epstein civil violence.ipynb
  • Forest Fire Model - forest_fire/forest fire model.ipynb

Start with the Forest Fire model - it is one of the best known introductory models in the field. As you experiment with these models, ask yourself, ‘what would it take for this to be a model of an archaeological concept?’


Computer Vision

Binder Repository

Contains:

  • Building an archaeological image classifier with tensorflow.ipynb

This notebook demonstrates how Transfer Learning can be applied to create a neural network model trained on archaeological imagery. A mobile version that uses the results of this notebook to create an image identification app may be followed here


Clustering Images with Tensorflow

Binder Repository

Contains:

  • find-similar-images.ipynb
  • Affinity Propogation.pynb

These notebooks take the second-last layer of the neural network and demonstrate how to study it for clustering visually similar images.


Sonification

Binder Repository

  • Intro to Sonification.ipynb

This notebook walks through the process of mapping time-series data to musical notation, to create mid files that can then be turned into sound.


Creativity

Binder [https://github.com/o-date/creativity/]

  • Glitching an image with prism sorting.ipynb
  • semantic_similarity_chatbot.ipynb

This binder can take quite a bit of time to pull together. Please be patient. There are steps in the chatbot notebook that can also take a bit of time; watch for the [*] beside a running code block to disappear before moving to the next one.


Creativity 2

Binder Repository

  • World Building - model demo.ipynb
  • History Generator - ahistorygenerator.ipynb

The first notebook generates a world by simulating topography and erosion, and then using an agent based model to play out its history. The second is a far more simple model of state formation, fision, and fusion but uses Tracery to generate its historical chronicle, and graphviz to visualize it.


Make a Research Compendium

Binder Repository

This repository is an experimental demonstration of how you might combine a research compendium created by rrtools with Binder, a service that creates an executable environment with RStudio in your browser.

Please read the rrtools documentation and this repository’s readme before launching this binder.