Over the past year, Kenway has continued to invest in its partnership with Snowflake through a variety of experiences and certifications such as Snowflake SnowPro Core. Snowflake is a cloud data warehouse that unites siloed data, discovers and securely shares data, and executes diverse analytic workloads. We believe there are many opportunities for our clients to reap the benefits of this innovative cloud warehouse and data lakehouse platform.
As one of Kenway’s certified Snowflake SnowPro Core employees, I recently had the opportunity to attend Snowflake’s two-day virtual summit, which provided me significant insights into all the new features they will soon be releasing to the public. Here are some of the overall themes and highlights that were discussed:
Every Snowflake user can access data across the cloud, regardless of region, which provides organizations a better means for data collaboration. Some companies are using Snowflake to bring in their data in real time, which allows them to get up-to-date insights on transactions. Snowflake has substantially increased the data sets available on its Data Marketplace. The Marketplace is where companies can share their data rather than make copies of it and then move it around. Another key topic in the Connected Industries domain was Snowflake’s new integration with ServiceNow. This native integration allows the ServiceNow data to be readily available in Snowflake for companies to manage consumer relationships and become part of a central data repository, rather than being siloed.
Another powerful feature in Snowflake that helps industries stay connected is its improved data sharing capabilities. There are three methods for organizations to share data: share it with other accounts, publish to a private exchange, or publish to the public marketplace. The new API will help guide users through the process to make entire databases, tables, views or functions available to those with whom they’d like to share. Snowflake sharing eliminates the need to create and build processes to move data inside or outside of an organization. There is one copy of the data created, and data updates are made available to consumers in real time.
Last year, Snowflake released a feature that allowed users to mask data dynamically. To add onto that, they have been working on row access policies and object tagging. Personally identifiable information (PII) is difficult and tedious to identify in data, and is usually done manually. Snowflake now has two key features that allow its customers to tag data automatically through classification and anonymized views. They can then apply functions to the table that will either generalize or suppress that data. These features still provide the analytical value needed to generate insights and protect customers’ information.
Security has also been front and center when talking about storing data in the cloud. To address this concern, Snowflake has a layered security model that includes network security, identity authentication and access management, and single sign-on. One of the latest improvements is the private connectivity to users’ Snowflake internal stages. The staged data that is accessed through client apps remains on the private network. On AWS, users access Snowflake’s S3 PrivateLink capability where the data stays on the Amazon private network which eliminates the need for proxies. Azure has a similar capability called Azure Private EndPoint. Another important security feature is the new session policies that can be set at the account level or individual users, as well as database and UI timeouts.
Snowflake continues to improve its performance and efficiency, and the most recent updates made are better compression and query acceleration. All new data being written to a warehouse is compressed even more, resulting in some customers benefiting from a 30% cost savings for user storage. The query acceleration service boosts performance which can result in 15 times performance improvement and less latency, giving users better predictability on query response times. These new features are being done behind the scenes so that there is no impact or downtime to Snowflake users. In conjunction with performance and storage improvements, users will soon have a new administrative experience with improved admin screens to better track usage, storage and costs for their organizations.
Snowflake users have been utilizing task scheduling to help with their data pipelines. One improvement to this feature is serverless tasks which allow for serverless execution, and automatically determine the appropriate amount of compute resources. Another new feature in this domain is schema detection which can determine the schema for semi-structured file types of Parquet, ORC or Avro. Users will soon also be able to store and process files as they do with structured or semi-structured data, eliminating the need to load data from files in the data lake. Developers will now have the ability to create functions and stored procedures in a SQL based language rather than just JavaScript.
Another new feature built to make developers’ lives easier was Snowflake’s SQL API. This allows them to submit SQL calls through the API, which supports standard queries, DDL and DML statements. These API calls are lightweight with little overhead. Why is this important? If organizations migrating their applications use Snowflake as their data warehouse, there will be no need for them to refactor the code since Snowflake already includes REST API. This key benefit ultimately helps reduce migration costs.
There were many more great discussions at Snowflake Summit 2021, all of which are now available to watch on-demand. We’ve already begun to share some of these insights with our clients, and look forward to helping them leverage these capabilities.
If you attended the Summit or have a chance to watch the recorded sessions, I encourage you to share your thoughts and favorite takeaways. Please connect with me on LinkedIn to discuss what you found most interesting – I’d love to compare notes!
