Data Warehousing
Whether it is starting up a new business and you need a scalable way to store data, or you are needing help managing your current data storage solution, I can help. Data is the fastest growing and one of the most essential commodities in business today. If your analysts are unable to get to the data they need, or use the data effectively to deliver key insights, it is probably due to a sub-optimized data warehouse. Whether it is stored procedures, indexing tables, or even well-defined triggers there are a lot of things that can be done to help optimize your relational database. Typically I have used MySQL, Postgres, MS-SQL when dealing with relational databases for clients. However, if possible I would strongly advise using Amazon's RDS as well.
There will be times that a traditional relational database is not sufficient, or it doesn't play well with your use cases. These use cases often lend themselves towards using alternative data storage methods such as NoSQL and streaming messages. For these types of projects I have most often used MongoDB (a great NoSQL backend that works well with Javascript applications). Other non-relational data storage solutions I have used include: Cassandra, Kafka, RedShift, Snowflake and even Redis (a caching solution).
Recommended Tools:
Mysql
- Stored Procedures
- Advanced User Interface
- Amazon Aurora Compatibility
These are three quick easy wins for why MySQL is a great tool that we could implement or enhance the existing implementation. It his highly rated by analysts as long as the data warehousing strategy was executed on effectively.
postgres
This is a great low-maintenance back end solution, with a good community supporting it's development. Now that is also supports upsert functionality, and is able to support a myriad of extensions, PostgreSQL is a fantastic option for storing your relational data. It integrates well with other applications and can scale fairly easily to support whatever application is attempting to store its data.
mongodb
This mLab blog gives the best breakdown for why MongoDB is great for modeling many of the entities that back most modern web-apps:
- Account and user profiles: can store arrays of addresses with ease
- CMS: the flexible schema of MongoDB is great for heterogeneous collections of content types
- Messaging: vary message meta-data easily per message or message type without needing to maintain separate collections or schemas
- System configuration: just a nice object graph of configuration values, which is very natural in MongoDB
- Log data of any kind: structured log data is the future
- Graphs: just objects and pointers - a perfect fit
- Location based data: MongoDB understands geo-spatial coordinates and natively supports geo-spatial indexing
Snowflake
For the best hands-off management of your data that can handle both structured and unstructured data, while still offering up the ability to query with SQL, Snowflake has proven its self time and time again. This warehouse solution scales really well and can accommodate a variety of use cases rather seamlessly.