Object Storage And Web Applications
Object storage is another way to store your data, objects are used as transactional units to store data along with metadata that can be extended. You can think of an object as a PDF file, BLOB bytes or a Text file (unstructured data).
the structure is not the same as the old file system where you store files inside folders (the tree structure). With object storage there are no folders inside folders, the concept is all about files stored as objects on a one level structure called the flat address space.
it’s wildly used in cloud computing. You interact with the data stored using HTTP requests (GET, PUT, …), and there are lots of software that communicates over HTTP from anywhere over the internet.
The main advantage of object storage is the fast read operations of objects and its distributed nature making it a very effective way to store unstructured data for any type of web based application. On the other hand, object storage becomes a hassle if you’re storing data that changes very often (think databases).
Web application static files (css, js, images …etc) and user generated files are best stored at an object storage backend, as changes are not very often, and being distributed (on some cloud service providers) makes it cached across many locations speeding up the requests from application users (with Content Delivery Network Support). And since object storage services allow data retrieval through HTTP protocol, you don’t need a static files server on your hardware to serve those files as when using a block storage solution. Object storage service providers are responsible for maintaining the physical layer (Hard Disks, RAID Configurations, …), this makes less work for developers to account for.
Some of the major providers of cloud object storage solutions are Amazon, Rackspace, Digitalocean and many more, you can create an account and buy an object storage solution, one big advantage is that you only get to pay for the space you use, much cheaper than buying block storage space upfront. You then get to manage access to the storage using access keys. And then deploy those keys to your application to read / write files.
Storing file directories to an object storage
Object storage solution uses a flat address space and doesn’t support working with nested files and folder. If you used Dropbox or Drive personal cloud storage solutions, they all use object storage on the backend (Dropbox uses Amazon S3 services), on Dropbox, for example you can nest files inside folders, this is done as a trick with the object names, for simple demonstration, if you upload a file to /foo/bar/file.txt, an object is created with the name set as the full path “foo-bar-file.txt”, it’s not exactly like this but you get the main idea.
Hosting object storage
You can go with two ways to host an object storage solution if you’re planning to use it in your application:
- Buying from cloud providers. You can enroll in an object storage plan on many providers like Amazon, Rackspace and Digital Ocean companies, they let you choose the datacenter location and other configurations like privacy. You can leave the maintenance to them and only care about connecting your application to the storage. This is often easier to scale.
- Self-hosting object storage. It is possible to host your own object storage on top of a block storage volumes you already own (or rent). You have to maintain the storage server, this gives you more privacy over using 3rdparty solutions. Minio is an open source server the you can setup on your server to host S3 compliant object storage server.
How it differs from block storage
Block storage technology uses typical hard disks connected to a server. The server operating system then creates multiple volumes (blocks) on those disks and use them as separate drives for all of your applications.
this adds flexibility with the created volumes. You can host a traditional file storage on one volume and a database storage on another. Each created block can be formatted with any file system type using the operating system. Think of each volume as a separate hard drive.
Blocks can be reconnected to a different server, and they are easy to resize too. However, the biggest advantage of using block storage over object storage is that they have faster IO transactions. for example, to back a database system you need a block storage device, because databases in general require fast connections to their storage backend when making reads and writes.
As a web developer, you use block storage drives on your server to store the main project files and to back the database.