Hosted Data and Algorithm Data

Hosted Data

Some Algorithms accept, and return, only moderately-sized amounts of data: for example, SentimentAnalysis accepts a piece of text which might be just a few words or several pages long, and returns a number.

But some Algorithms work on binary data, or on very large amounts of information. In these cases, it is necessary to send and/or receive a file.

Take a look at the sample input and output for the ColorfulImageColorization Algorithm:

Note the File URLs on the left side which begins with "data://". This indicates that the input to the Algorithm will be a file, specifically one found inside Algorithmia's Hosted Data: a place where you can upload and store files.

To put a file into Hosted Data, head to https://algorithmia.com/data and click on "My Hosted Data". This lists all your Data Collections -- basically, folders you can put files into. Click on the "New Collection" button to the upper-right to create a Collection. Take note of the URL underneath your new Collection's name; in this example, my username is "demo" and the collection is called "example_data", so the folder's URL is "data/demo/example_data".

Click on this Collection. Initially, it contains no files, but take note of the "Read Access" near the top of the page. When this is set to "Private", only you can view these files. If you change this to "Public", any Algorithmia user can read your files -- only do this for things you intend to share, like demo files. The default option, "My Algorithms", is the same as "Private" until/unless you decide to create an Algorithm of your own (at which point, those Algorithms have access to your Collection).

Let's upload a file. You can drag-and-drop a file from your computer onto this page, or click the "Upload Files" button in the upper-right.  Once your file is uploaded, you'll see a URL appear under its name ("data://demo/example_data/image.jpg" in the example below). This is the way you'll refer to the file when passing it to an Algorithm.

Try it out by clicking the ellipsis (...) to the right of the file, and picking "Copy URL". Then, go to ColorfulImageColorization and paste the URL into the Input under "Run an Example" (making sure to surround it with double-quotes). Click Run, and it will colorize your image for you!

One final note on Hosted Data: at the beginning of data URLs, ".my" can be used as an alias for your own username. So, if my username is "demo", both "data://demo/example_data/image.jpg" and "data://.my/example_data/image.jpg" refer to the same file.  More info in the Data API Documentation.


Algorithm Data

Take another look at the "Output" side of the image colorization example:

Algorithms have the ability to save files as output. When they do, they put them into Algorithm Data, which you can find by going back to  https://algorithmia.com/data and clicking on "Algorithm Data". Here, you'll see a Collection for each Algorithm you've run which creates files.

Clicking on any of these, note that there are two sections in each Collection: "Files" and "Temporary Files". Most Algorithms place their output under "Temporary Files", which are deleted after a day, so be sure to download any files you want to retain permanently.

Note that the URLs of these Collections follow a specific format: "data://.algo/", followed by the Algorithm's name ("deeplearning/ColorfulImageColorization" in this example), then either "/temp/" or "/perm/" depending on whether they are temporary or permanent files.  For more info, see the Data API Documentation.


The web interface isn't the only way to move files around -- we also have a full Data API for creating, reading, and deleting files and folders.

But first, let's learn how to connect to other data services.

 


NEXT