among the problems we have actually noticed with ETL pipelines is easily getting information from spreadsheets into information warehouses.
lots of businesses still have a great deal of data in spreadsheets that don’t appear like a database table with rows and columns.
on the other hand, there are information engineering groups that are constructing extremely powerful information pipelines that require that data be in a relational format.
right now – to get spreadsheet data into a data storage facility like Snowflake, information engineers have to write their own ETL code for spreadsheets or use something like stitch information’s google sheets combination to get the information into the data warehouse.
nevertheless, this is mistake susceptible because there is no immediate feedback to the business user when they make edits to the spreadsheet – it just calmly fails later on with an error in the logs when you are attempting to extract the data.
it likewise requires the spreadsheet user to format their spreadsheet to appear like a relational table (which is frustrating for financing and operations users simply wanting to utilize a spreadsheet) and they need to keep an eye on psychologically where they can or can not modify the spreadsheet.
so – to solve this problem, we developed structure sync, an easy method to setup spreadsheet ETL within google sheets so that service users can modify spreadsheets with instant feedback on whether they will sync properly and information engineers can quickly build a transform pipeline that turns any spreadsheet into information that looks like a relational table.
we presently are waiting on google to authorize our app – so please let us understand if this is a problem in the comments and register on our sign up page!
we are also taking a look at supporting microsoft excel and other information warehouses, let us know in the comments if this is something you ‘d have an interest in.
Inspect us out @ http://sync.structure.rest