The DSA Learning Environment

Last updated on 

Our Data Science Learning Environment

The learning environment has the following key components:

  • JupyterHub, the DSA program interactive web application for real-time data science learning, coding, visualization, etc.
  • GitLab, the DSA program centralized Git repository and course learning activity distribution server.

DSA Logical Cyberinfrastructure

The local DSA cyberinfrastructure has a few components that you will see referenced in the various learning activities. You will become accustomed to these components and usage, and the summary here is provided for context relative to JupyterHub discussions. JupyterHub, as mentioned, is the centralized web application that allows students to complete learning activities.

Overview and Motivation

As part of the Data Science and Analytics program at MU, you will utilize a wide collection of computing resources, tools, and techniques in your various courses. These include the following:

  • Python programming language, libraries, and ecosystems
  • R statistical programming language, libraries, and ecosystems
  • Databases of many varieties including relational (SQL), NoSQL, graph, and spatiotemporal
  • Cluster and cloud computing

The MU JupyterHub environment will serve as the central access point for utilizing all the computing resources referenced above. This environment provides access to Python, R statistical programming, and Terminal (computing server) access. Together, these three technologies provide mechanisms for API-based access and data processing orchestration on computing resources such as the cloud and cluster computing environments. Other resources such as AWS, Google Cloud, and the MU NSF Supercomputer will provide hosting for massive data sets and cloud/cluster computation, and will be accessed via the JupyterHub environment.

For example, some learning activities in some DSA courses will be Python notebooks in JupyterHub which will utilize the GCloud python API to execute computational operations on Google Cloud's resources. As another example, the Google BigQuery API for R statistical programming will be incorporated into R notebooks, where Google BigQuery is a data source/repository and data query interface. In both cases, students will be logged into and interacting with the MU JupyterHub environment directly. Connectivity to the other cyberinfrastructure components originate from this MU resource via the API. Similar APIs exist between Python and AWS – for example, the BOTO3 API. Additionally, various learning activities will guide students through creation and submission of data processing work for supercomputing clusters via Python scripting and terminal commands.

We have also developed a tight integration and course module management system between JupyterHub and GitLab for the DSA program.

GitLab

GitLab is a centralized deployment of Git Version Control System repositories.

Git

Version control systems (VCS) are utilized as change management systems for a variety of electronic items, such as source code, configuration files, data files, documentation, etc. Git is a very powerful VCS that is rapidly expanding in use through the technology realms. Read more about Git here. Instead of completing lessons on Git and version control, you will learn it by continual use throughout the program.

GitLab and course modules

DSA courses are organized in a modular format within GitLab host repositories, which are automatically cloned for students to have their dedicated and persistent learning resources.

All courses adhere to this modular design, ensuring the cloned learning activities are available to students for the entirety of the student's program. One benefit is that all coursework becomes an online, personalized reference collection for each student.

The below diagram shows the deployment of course materials from the intructor to you, the student, and how you submit. Detailed instructions for submitting your completed notebooks and obtaining the next module are provided in the Using Git post and the Using Git video.

Need more information or help? Reach out to us at 📧 jcwdw@missouri.edu