Getting started with Python and IPFS

IPFS

In this post I am going to discuss how you can use  decentralized IPFS in your Python apps for storing different kind of data.

What is IPFS?

From Wikipedia:

InterPlanetary File System (IPFS) is a protocol and network designed to create a content-addressable, peer-to-peer method of storing and sharing hypermedia in a distributed file system. IPFS was initially designed by Juan Benet, and is now an open-source project developed with help from the community.

In simple it’s Amazon S3 on Blockchain. All of your information is available on decentralized network across nodes thus not only make the system scaleable but reliable as well since data which is fed in it will be there hence no chance of 404-File not found kind of errors.

So there can be many possibilities how you can use it, essentially for file storage, you can use as web hosting, for messaging system, possibilities are endless.

Installation

Download the setup file from here. Since I am on OSX so I just unpacked go-ipfs_v0.4.13_darwin-amd64.tar.gz and executed the file install.sh which will transfer the binary file in /usr/local/bin

To make sure everything is setup properly, run ipfs --version or ipfs on your console. If all set then you can see something like below:

➜ IPFS ipfs --version

and it prints

ipfs version 0.4.13

OK it’s installed. Now, it’s time to initialize your local IPFS node. For that run the following command:

ipfs init
When runs it prints following:

 

And when you run ipfs cat /ipfs/QmS4ustL54uo8FzR9455qaxZwuMiUhyvMcX9Ba8nUH4uVv/readme on command line it should show you something like below:

IPFS Installed

 

So, basically ipfs cat shows the content of the file. If you do ipfs get it will download it locally on your machine.

IPFS Daemon

When you run the command ipfs daemon it will start a node on your machine which will connect with other nodes available. The concept is similar to Blockchain that there are many nodes available and you start your own node which syncs with other nodes. If you want you can have a setup of private nodes by setting IPs in Swarm and Bootstrap section. For instance these are defaults:

 

you can view it when you run ipfs config show

So, when you do ipfs daemon it shows following output:

You can have a web interface by visiting http://localhost:8080/ipfs by adding Hash of the file object. So for instance when you run ipfs refs local it prints the content exist on your local node:

 

And you can view the content on your browser by doing  http://localhost:8080/ipfs/QmY5heUM5qgRubMDD1og9fhCPA6QdkMp3QCwd4s7gJsyE7 on your browser and can view the content. In my case it is  the help file by IPFS itself.

Pinning

When you post a file on IPFS network, it’s spread across the connected nodes. IPFS provides garbage collection concept which eventually let the files get deleted. Or, if your node goes offline and not synced the data will be lost. In order for your data stay there forever, you pin your file by running the command:

ipfs pin add <hash_of_content> where hash of content are the entries you see above. When you do it, it’s not removed due to garbage collection process. You can also unpin by running ipfs pin rm <hash_content>

So far we see how you can interact with IPFS via command line but how to automate it or even use in your python, let’s get into the code!

Development

I am not doing anything fancy, just calling commands we called via CLI. By learning that you can come up with your own Image Hosting Service or Messaging App or your own decentralized Amazon S3!

First install IPFS Python API, run the command pip install ipfsapi

Start local daemon by running ipfs daemon and create a local Python file in your favorite IDE.

If connected you will see ipfsapi.client.Client object at 0x104af5a90 otherwise exception like: ConnectionError: HTTPConnectionPool(host='127.0.0.1', port=5001): Max retries exceeded with url: /api/v0/version?stream-channels=true (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 61] Connection refused',))

OK, it’s connected now let’s add a new file:

new_file = api.add('new.txt')

It will add a new file. If you get a print() of it you can see a return dict object like below:

{'Name': 'new.txt', 'Hash': 'QmWvgsuZkaWxN1iC7GDciEGsAqphmDyCsk3CVHh7XVUUHq', 'Size': '28'}

Now let’s access this file via browser by visiting http://localhost:8080/ipfs/QmWvgsuZkaWxN1iC7GDciEGsAqphmDyCsk3CVHh7XVUUHq/

and it will show the newly added file.

In order to see the content of file you call api.cat('QmWvgsuZkaWxN1iC7GDciEGsAqphmDyCsk3CVHh7XVUUHq') and it will print content of the file, same thing you do via command line. You can learn more about IPFS and Python integration here.

Conclusion

IPFS is a new way to implement web, a new protocol that makes thing scaleable and reliable. You could consider it as a decentralized storage system running on blockchain. As a Python developer you can integrate new or existing  Python apps by using these simple to use APIs. Like always, code is available on Github

If you like this post then you should subscribe to my blog for future updates.

* indicates required