Case Study & Capstone Team Datashares

Last updated on 

As a student in the Case Study or Capstone course, your team has been provided with a team file share location to store all your data files, both source and derived data files. Your instructor will have provided you with the correct path to use.

Data files are not to be committed to your shared repository.

Method 1: Terminal Commands

You may use the scp, sftp, or rsync protocols to upload to your team file share on lz.dsa.missouri.edu. Instructions for scp are below.

Uploading a file or files using scp

  • Open a command window on your local computer or Jupyterhub.
    NOTE: if you are using your local computer, you will need to be on campus or University VPN.
    • Jupyterhub: Open a terminal as you normally do
    • Windows: type Powershell in the search box and open the app
    • Mac: Open your Applications folder, then open Utilities and double-click on Terminal.
    • Linux: If you are using Linux, you probably are familiar with the command line. You can usually open a command window by pressing Ctrl+Alt+t at the same time.
  • Navigate to the location where your file is located using the cd command.

To copy a single file

To copy a single file, execute the following command on one line, with no line return. After entering the command, you will be prompted for your university password. When you type in your password nothing will be displayed. Press enter when you are done entering your password. 

scp <local_data_file_path> yoursso@lz.dsa.missouri.edu:/dsa/groups/<team directory>/<datafilename>

For instance, if my sso (userid) is ttiger, the course directory is casestudy2022su, my team directory is team10, and the data file I want to copy is “data.csv” and I want to copy it using the name “data_source1.csv”: 

scp data.csv ttiger@lz.dsa.missouri.edu:/dsa/groups/casestudy2022su/team10/dat a_source1.csv

If you want to copy the file without changing its name, you can leave off the file name in the destination:

scp data.csv ttiger@lz.dsa.missouri.edu:/dsa/groups/casestudy2022su/team10/

To copy a directory

To copy a directory, execute the following command (the difference from the command above is the -r flag.) You will be prompted for your university password.

scp -r <local_data_directory_path>
yoursso@lz.dsa.missouri.edu:/dsa/groups/<coursedirectory>/<team
directory>/<directoryname>

For instance, if my userid is ttiger, the course directory is casestudy2022su, my team directory is team10, and the data directory I want to copy is “datadirectory” and I want to copy it using the name “datadirectory1”:

scp -r datadirectory
yoursso@lz.dsa.missouri.edu:/dsa/groups/casestudy2022su/team10/datadirectory1

If you want to copy the directory without changing its name, you can leave off the directory name in the destination:

scp -r datadirectory
yoursso@lz.dsa.missouri.edu:/dsa/groups/casestudy2022su/team10/

Check access

Your teammates should be able to access the file you uploaded in notebooks using its path. 

For instance: pd.read_csv(‘/dsa/groups/<team directory>/<datafilename>’) 

For example: pd.read_csv(‘/dsa/groups/casestudy2022su/team10/data.csv’)

Use the instructions below to view your file and check its permissions if your team has trouble accessing this file.

Checking and changing file and directory permissions

  • Make sure you are on campus or campus VPN for this, or using a terminal from Jupyter.
  • SSH into the lz server from your local or Jupyter terminal. You will be asked to enter your university password. When you type in your password nothing will be displayed. Press enter when you are done entering your password.
ssh yoursso@lz.dsa.missouri.edu
  • Change the directory to your team share
cd /dsa/groups/<coursedirectory>/<team directory>

# For example:
cd /dsa/groups/casestudy2022su/team10
  • List the files using the ls command with the -l flag to see details. See the ls man (manual) page for other flags for this command.
ls -l

Understanding permissions

Permissions will be listed in three groups of three letters (r for read, w for write, x for execute) or hyphens (for no permission for that function,) preceded by a d for a directory or a hyphen for a file. The first group of three letters is for the user owner of the file (the permision who made the file or directory); the second group of three letters is for the group owner of the file (we aren't using Linux groups at this time), and the third group of three letters is for all others (this will cover your team members.) 

# a directory with full permissions for everyone would be:
drwxrwxrwx

# file with read and write for the owner and read for everyone else would be:
-rw-r--r--

After the permissions, the user owner and the group owner will be listed. Most likely these will be the same and will be the person who made the directory or uploaded the file.

When setting permissions, numbers are usually used.

  • r (read): 4
  • w (write): 2
  • x (execute): 1

These are then added together to get the appropriate value. For instance, 4 is read, 6 is read and write, and 7 is read, write and execute.

  • To change permissions on a file to the proper permissions on a file or directory from inside its folder location the owner should run this command:
# for a directory
chmod 777 <directory name>

# for example
chmod 777 raw_data

# for a file
chmod 444 <file name> # everyone has read access
chmod 666 <file name> # everyone has read & write access

# For example:
chmod 744 data.csv
chmod 766 data.csv

Working with your files

Files can be removed with the rm command, copied with the cp command, and moved or renamed with the mv command. Directories can be created with the mkdir command. In each case, keep an eye on the permissions. You will need to set the permissions if you use mkdir to create a directory.

Method 2: Client Application

There are various client applications to assist with file transfers using the above protocols. I will provide an example using the Winscp client on a Windows machine, which is what I personally use.

  • Download and install Winscp: https://winscp.net/eng/index.php
  • Make sure you are on the campus network or using VPN for this step.
  • In Winscp, set up a session using the New Site option for the lz.dsa.missouri.edu server using the sftp protocol, port 22, and your university single sign-on (userid) and your university password and save it for future use giving the session a name of your choosing. Highlight that named session and click the login button to connect using that session.
  • In the left-hand panel, navigate to the folder location on your machine where the file or directory you wish to move is located.
  • In the right-hand panel, there is a drop down box above the visible folders and files that lets you easily navigate to the correct folder. Navigate to root/dsa/groups/<coursedirectory>/<teamdirectory>
  • You can drag and drop the data file or folder from left to right.
  • Look at the file permissions on the right, under the “Rights” column. Make sure that others have read permissions.
  • To change the permissions on the file, right click it and open Properties. Check the boxes for R for Owner, Group, and Others, or R and W for each of them if the file should be able to be edited.

Need more information or help? Reach out to us at 📧 jcwdw@missouri.edu