Skyplane: 110x faster data transfers on any cloud- 1 min
Skyplane: 110x faster data transfers on any cloud
At Sky Lab, I am working on the Skyplane project, a tool for blazingly fast bulk data transfers in the cloud. Here are my contributions so far.
I showed that Skyplane is up to 110x faster and 84% lower cost than any existing cloud transfer tools such as AWS DataSync, Google Cloud Data Transfer, and Azure Data Transfer.
I implemented a real-time dashboard of the public cloud internet activities related to Skyplane. Below are two of the plots (up to 12/01/2022).
Training an ML model with Skyplane
After implementing Skyplane API, I designed a use case of Skyplane about transfering data between S3 buckets and streamlizing it directly to the ML model.
Detailed instructions are here.