In Hosted Data, we covered the basics of managing files in Algorithmia. Now that you know how to work with files and collections (folders) through the web interface, let's take a look at the Data API , which lets us do the same thing in code.
Creating a Client Instance
Just as we need an Algorithmia Client to be able to call Algorithms, we need to create a Client instance to manipulate files in our account. In Python, this would be:
import Algorithmia
client = Algorithmia.client("YOUR_API_KEY")
Working with Collections / Folders
Whether or not it exists yet, we refer to a collection or subfolder via client.dir, passing it a Data URL ("data://", "s3://", etc). Then, we check to see if it exists, and if not, create it:
mydir = client.dir("data://somecollection")
if not mydir.exists():
mydir.create()
We can also list the files or subdirectories in a dir:
for myfile in dir.files():
print(file.path)
Reading Files from Data Collections
While we can't directly work on files in Data Collections, we can refer to them (whether or not they exist yet) via client.file, passing a Data URL:
myfile = client.file("data://somecollection/somefile.txt")
If that file already exists, we can retrieve its contents as text or bytes:
if myfile.exists():
text_content = myfile.getString() #for text files
binary_content = myfile.getBytes() #for non-text
Or, we can copy that file from the Data Collection to a local directory:
localfile = myFile.getFile()
This last operation is especially important, because client.file("data://...") is not a language-native File object -- for example, you can't open() it like a normal Python file handle. But myfile.getFile() does give you a real file handle, because it copies the file from the Data Collection to a local directory on your computer. Then, you can do normal file operations like:
f = open(localfile.path, "r")
for line in f:
print line
Writing Files to Data Collections
Again, client.file does not give us a true filehandle -- so we can't write to it directly with language-native operations like f.write -- however, we can copy a local file into a Data Collection, like so:
client.file("data://somecollection/newfile.txt").putFile("/tmp/localfile.txt")
Here, we're copying a file from our local computer ("/tmp/localfile.txt") up to a Data Collection ("data://somecollection/") where it will be written as a new file ("newfile.txt"). If a file with that name already exists inside "data://somecollection/", it'll be overwritten.
There is one shortcut to be aware of: if you just want to create a text file in a Data Collection, you can directly write a string to a client.file:
client.file("data://somecollection/newfile.txt").putFile("Some text I want to put in the file.")
Many more operations, such as client.file("data://...").delete() are available: see the complete Data API Specification for more functions and examples in a variety of programming languages.
If you are in the middle of the "Using Algorithms & Extending your Apps" series, continue onward to Learn How Credits Work.