Ionburst Cloud offers a revolutionary way to store data securely and privately in the Cloud, beyond the reach of hackers and unwanted surveillance. Data is transformed and persisted as redundant fragments across collections of storage nodes called Cloudlets™.
- An active Ionburst Cloud subscription. Get started for free.
- The GitLab repository for the Ionburst Send project is available here.
In this tutorial, we will integrate Ionburst Cloud as the back-end storage for the open-source file-sharing application, Firefox Send.
Firefox Send is a free, end-to-end encrypted file-sharing application developed by Mozilla, that allows users to easily and safely share files over the Web. The Send back-end is written in Node.js, allowing us to integrate with the Ionburst Node.js SDK.
Note: The public Firefox Send instance was temporarily taken down on the 7th July, due to reports it was being abused by malicious actors sharing malware. This appears to have also had an impact on the stability of the code base on Github. For this reason, we have based our Ionburst Cloud integration on an older, stable commit that we found to work – available here.
Digging Into the Send Source Code
From a cursory review of the source code and running the application locally, it looked like the focus of our integration would be the
server directory, which houses the code for Send’s back-end services.
Of most interest was the
storage sub-directory, which contains the functionality for integrating Send with the following:
- Local filesystem storage;
- Google Cloud Storage;
- Amazon S3;
A review of these files outlined common pieces of functionality expected from storage integrations:
length– returns the object size from the configured storage method;
getStream– retrieves the object from the configured storage method;
set– uploads or writes the object to the configured storage method;
del– removes the object from the configured storage method;
Of these, Ionburst Cloud provides functionality for three out of four, as it does not expose the ability to query object size. Further digging suggested that the
length function was being used to set the
Content-Length header on file download to the user, so wasn’t a functional requirement for the storage integration.
Exploring the storage sub-directory also confirmed how Send handles object metadata. In development, Send uses a local, in-memory store to track each object but is designed to use Redis in production. To gain a better understanding of the Send application, all Ionburst Cloud integration work was carried out using Redis as the Send metadata store.
The final checks required were to see how the Send back-end handled storage configuration. The base back-end configuration is handled in the
config.js file found in the server directory, which defines which storage method is selected by the
index.js file found in the
Integrating Ionburst Cloud – Configuration
To begin integrating Ionburst Cloud with Send, we first have to add new configuration options to the Send project to tell it to use Ionburst Cloud as the new storage method, along with the initial Ionburst SDK configuration.
The Ionburst SDK was added to the project using
A local Redis instance is deployed to track Send metadata using Docker:
config.json file is added to the root of the Send project for the Ionburst Cloud config file.
A new configuration item is added to the Send
config.js file for Ionburst Cloud. Note: this configuration entry is only used to select Ionburst Cloud as the chosen back-end storage, and does not perform any other configuration. An entry is also added to ensure Redis was used in development:
A configuration option is added to the storage
index.js file, to ensure Ionburst Cloud was selected as the storage method:
ionburst.js file is created within the
storage sub-directory, and a constructor created for applicable configuration:
Integrating Ionburst Cloud – File Operations
Ionburst Cloud PUT
From the storage
index.js file, we can see how Send kicks off a file upload to its configured storage:
From this, we can see that Send generates an identifier for each file stored, before passing it and the file to the configured storage method. As Ionburst Cloud doesn’t care how a given object is identified, we can simply pass this identifier to Ionburst Cloud too.
To upload the data to Ionburst Cloud, the following function was created in
We encountered some issues passing the file object directly to the Ionburst SDK. Instead we leveraged the existing filesystem functionality to write the file to a temporary directory, creating a read stream for the Ionburst SDK, then removed the temporary file after successful upload.
This temporary file/directory leveraged functionality used by Send’s file-system storage, and it was simply a matter of pulling the temporary directory configuration into the Ionburst Cloud storage constructor:
Ionburst Cloud GET
Similar to the upload function, the main download functionality can be found in the storage
To keep things simple, we replicate the same temporary file functionality for the file download from Ionburst Cloud:
We first grab the file from Ionburst Cloud, write it to the temporary directory, and create and return a read stream.
Ionburst Cloud DELETE
Send requires delete functionality from the configured storage method to remove uploaded files once they have reached their download limit, or expiry time.
The Ionburst Cloud delete function was simple to implement:
Integrating Ionburst Cloud – Final fixes
With the basic upload, download and delete functionality for Send’s integration into Ionburst Cloud, it was a simple case of tidying things up.
Ionburst Cloud Configuration issues
The Ionburst SDK for Node.js uses two methods to obtain its configuration, a
config.json file to be kept in the root of the project and credentials file located in the user’s home directory, or by environment variable.
As we don’t typically develop in Node.js, Send is the first project we’ve encountered that uses Webpack. While beginning the Ionburst Cloud integration, we found that Webpack interferes with the methods used by the Ionburst SDK to find the root directory of the project, namely:
There are two workarounds for this issue. Ionburst Cloud credentials can instead be specified by environment variable, and the Ionburst Cloud API URI specified on import:
Or, the Ionburst SDK settings method can be modified as such:
While this method works, we haven’t yet pushed this change to the live SDK, so we can evaluate the impact of the change. This issue is currently being tracked here on Gitlab.
Any feedback from the Node.js community is much appreciated.
As Ionburst Cloud has no method of returning a stored object’s size, and the Send metadata doesn’t track it either; a different method needs to be found to attach the object size to the final download request.
Content-Length is set server-side in
filelist.js but removing the original method doesn’t appear to have an effect on functionality.
The publicly hosted version of Send has a file size limit of 1GB, or 2.5GB for registered users. Ionburst Cloud currently supports a maximum object size of 50MB, with larger objects requiring client-side processing before upload.
As a simple proof-of-concept, we’ve kept this limit in place – allowing larger objects to be uploaded to Ionburst Cloud by Firefox Send would require additional development beyond the scope of this tutorial.
Integrating Firefox Send with Ionburst Cloud was a quick and simple process. It provided an opportunity to try out our Node.js SDK in practice. There were some intricacies to be handled, but it allowed us to demonstrate how Ionburst Cloud can be integrated to secure the data of existing applications.