The Data API

In Hosted Data, we covered the basics of managing files in Algorithmia. Now that you know how to work with files and collections (folders) through the web interface, let's take a look at the Data API , which lets us do the same thing in code.


Creating a Client Instance

Just as we need an Algorithmia Client to be able to call Algorithms, we need to create a Client instance to manipulate files in our account. In Python, this would be:

import Algorithmia
client = Algorithmia.client("YOUR_API_KEY")


Working with Collections / Folders

Whether or not it exists yet, we refer to a collection or subfolder via client.dir, passing it a Data URL ("data://", "s3://", etc). Then, we check to see if it exists, and if not, create it:

mydir = client.dir("data://somecollection")
if not mydir.exists():
    mydir.create()

We can also list the files or subdirectories in a dir:

for myfile in dir.files():
    print(file.path)


Reading Files from Data Collections

While we can't directly work on files in Data Collections, we can refer to them (whether or not they exist yet) via client.file, passing a Data URL:

myfile = client.file("data://somecollection/somefile.txt")

If that file already exists, we can retrieve its contents as text or bytes:

if myfile.exists():
    text_content = myfile.getString() #for text files
    binary_content = myfile.getBytes() #for non-text

Or, we can copy that file from the Data Collection to a local directory:

localfile = myFile.getFile()

This last operation is especially important, because client.file("data://...") is not a language-native File object -- for example, you can't open() it like a normal Python file handle. But myfile.getFile() does give you a real file handle, because it copies the file from the Data Collection to a local directory on your computer. Then, you can do normal file operations like:

f = open(localfile.path, "r")
for line in f:
    print line


Writing Files to Data Collections

Again, client.file does not give us a true filehandle -- so we can't write to it directly with language-native operations like f.write -- however, we can copy a local file into a Data Collection, like so:

client.file("data://somecollection/newfile.txt").putFile("/tmp/localfile.txt")

Here, we're copying a file from our local computer ("/tmp/localfile.txt") up to a Data Collection ("data://somecollection/") where it will be written as a new file ("newfile.txt"). If a file with that name already exists inside "data://somecollection/", it'll be overwritten.

There is one shortcut to be aware of: if you just want to create a text file in a Data Collection, you can directly write a string to a client.file:

client.file("data://somecollection/newfile.txt").putFile("Some text I want to put in the file.")


Many more operations, such as client.file("data://...").delete() are available: see the complete Data API Specification for more functions and examples in a variety of programming languages.

If you are in the middle of the "Using Algorithms & Extending your Apps" series, continue onward to Learn How Credits Work.

 


NEXT