Google Cloud | What is Bigtable

How BIG Is Cloud Bigtable?

“Building an application that needs low latency and high throughput?—You need a database that can scale for a large number of reads and writes. Cloud Bigtable is designed to handle just that. 

Cloud Bigtable is a fully managed wide-column NoSQL database that scales to petabyte-scale. It’s optimized for low latency, large numbers of reads and writes, and maintaining performance at scale. It offers really low latency of the order of single-digit milliseconds. It is an ideal data source for time series and MapReduce-style operations. Bigtable supports the open-source HBase API standard to easily integrate with the Apache ecosystem including HBase, Beam, Hadoop and Spark. It also integrates with Google Cloud ecosystem including Memorystore, BigQuery, Dataproc, Dataflow and more. 

Google Cloud | What is Bigtable

Some Cloud Bigtable Features

  • Data is by default encrypted with Google managed encryption keys but, for specific compliance and regulatory requirements if customers need to manage their own keys, customer managed encryption keys (CMEK) are also supported.
  • Bigtable backups let you save a copy of a table’s schema and data, then restore from the backup to a new table at a later time. Backups can help you recover from application-level data corruption or from operator errors such as accidentally deleting a table.

Scale and High Availability (HA)

How BIG is Bigtable? Bigtable has nearly 10 Exabytes of data under management. 

It delivers highly predictable performance that is linearly scalable. Throughput can be adjusted by adding/removing nodes — each node provides up to 10,000 operations per second (read and write). You can use Bigtable as the storage engine for large-scale, low-latency applications as well as throughput-intensive data processing and analytics. It offers high availability with an SLA of 99.9% for zonal instances. It’s strongly consistent in a single cluster; replication between clusters adds eventual consistency. If you leverage Bigtable’s multi cluster routing across two clusters, the  SLA increases to 99.99% and if that routing policy is utilized across clusters in 3 different regions you get a 99.999% uptime SLA.

Replication for Cloud Bigtable enables you to increase the availability and durability of your data by copying it across multiple regions or multiple zones within the same region. To use replication in a Bigtable instance, just create an instance with more than 1 cluster or add clusters to an existing instance. Bigtable supports up to 4 replicated clusters located in Google Cloud zones where Bigtable is available. Placing clusters in different zones or regions enables you to access your data even if one zone or region becomes unavailable. Bigtable treats each cluster in your instance as a primary cluster, so you can perform reads and writes in each cluster. You can also set up your instance so that requests from different types of applications are routed to different clusters. The data and changes to data are synchronized automatically across clusters. 

How does it optimize throughput

Through separation of processing and storage, Cloud Bigtable is able to automatically configure throughput by adjusting the association of nodes and data. In the rebalancing example, if Node A is experiencing a heavy load, the routing layer can move some of the traffic to a less heavily loaded node, improving overall performance. Resizing comes into play when a node is added to again ensure a balanced load across nodes, ensuring best overall throughput.

Click to enlarge

Choice of app profile and traffic routing can also affect performance. An app profile with multi-cluster routing automatically routes requests to the closest cluster in an instance from the perspective of the application, and the writes are then replicated to the other clusters in the instance. This automatic choice of the shortest distance results in the lowest possible latency. An app profile that uses single-cluster routing can be optimal for certain use cases, like separating workloads or to have read-after-write semantics on a single cluster, but it will not reduce latency in the way multi-cluster routing does.

Replication can improve read throughput, especially when you use multi-cluster routing. And it can reduce read latency by placing your data geographically closer to your users.  Write throughput does not increase with replication because write to one cluster must be replicated to all other clusters in the instance. Resulting in each cluster spending the CPU resources to pull changes from the other clusters.

Conclusion

Bigtable is a database of choice for use cases that require a specific amount of  scale or throughput with strict latency requirements, such as IoT, AdTech, FinTech, gaming and ML based personalizations. You can ingest 100s of thousands of events per second from websites or IoT devices through Pub/Sub, process them in Dataflow and send them to Cloud Bigtable. For a more in-depth look into Cloud Bigtable check out the documentation or join the upcoming webinar with our experts Build high-throughput, low-latency apps with Cloud Bigtable.

For more #GCPSketchnote, follow the GitHub repo. For similar cloud content follow me on Twitter @pvergadia and keep an eye out on thecloudgirl.dev.

By: Priyanka Vergadia (Developer Advocate, Google)
Source: Google Cloud Blog



For enquiries, product placements, sponsorships, and collaborations, connect with us at hello@globalcloudplatforms.com. We'd love to hear from you!


Our humans need coffee too! Your support is highly appreciated, thank you!

Total
0
Shares
Previous Article
Google Cloud | Cloud Web Publishing | Clouds

Google Cloud VMware Engine Now HIPAA Compliant

Next Article
Google Cloud | AI ML

New Research Reveals What’s Needed For AI Acceleration In Manufacturing

Related Posts
Read More

Call for Code Names Winner of 2020 Global Challenge and Announces New Initiative to Combat Racial Injustice

Agrolly selected for innovative solution to help small farmers as they struggle with effects of climate changeNew initiative enables global developers to leverage proven Call for Code platform to help combat systemic racism ARMONK, N.Y., Oct. 13, 2020 /PRNewswire/ -- Where most people see challenges, developers see possibilities. That's certainly been the case in 2020. Today, Call for Code founding partner IBM (NYSE: IBM) and its creator, David Clark Cause, announced the winner of the 2020 Call for Code Global Challenge. The top prize went to Agrolly, an application to help the world's small farmers cope with the environmental and business challenges of climate change.   Call for Code also introduced a new initiative—Call for Code for Racial Justice—to urge its international community of hundreds of thousands of developers to contribute to solutions to confront racial inequalities. The announcements came during a virtual event, the "2020 Call for Code Awards: A Global Celebration of Tech for Good." A full replay can be watched here. The 2020 Call for Code Global Challenge had asked developers to create solutions to help communities fight back against climate change and COVID-19. A panel of industry leaders and judges awarded Agrolly the grand prize while announcing four other winners—one that also created a response to climate change, and three others aimed at the global coronavirus pandemic. Agrolly will receive $200,000, support from IBM Service Corps and technical experts, and ecosystem partners to incubate, test and deploy their solution. Agrolly will also receive assistance from The Linux Foundation to open-source their application so developers across the world can improve and scale the technology. Call for Code unites hundreds of thousands of developers to create and deploy applications powered by open source technology that can tackle some of the world's biggest challenges. Since its launch in 2018, this movement has grown to more than 400,000 developers and problem solvers across 179 nations, and has generated more than fifteen thousand solutions using technology including Red Hat OpenShift, IBM Cloud, IBM Watson, IBM Blockchain, data from The Weather Company, and APIs from ecosystem partners like HERE Technologies and IntelePeer. Top Solutions Tackling Effects of Climate Change and COVID-19 Agrolly was created by a distributed team of developers from Brazil, India, Mongolia and Taiwan who met at Pace University in New York City. Powered by IBM Cloud Object Storage, IBM Watson Studio, and IBM Watson Assistant, Agrolly aims to fill information gaps so farmers with limited resources can make more informed decisions, and obtain the necessary financing to help improve their economic outcomes. By combining weather forecasts from The Weather Company and historical data from NASA with crop requirements published by the United Nations Food and Agriculture Organization, Agrolly's platform provides tailored information for each farmer by location, crop type and even the plants' stage of development during the growing season. The Agrolly team, as part of their response to the Call for Code Challenge, has made the solution available as an app in the Google store, free of charge. Another climate change solution, OffShip, received fifth place and was awarded $10,000. Three COVID-19 solutions were also honored. Second place went to Business Buddy, which will receive $25,000. Safe Queue was given third place and $25,000; SchoolListIt was awarded fourth place and $10,000. Safe Queue, an app that enables a safer way to manage entry during COVID-19 at shopping centers, small businesses, and polling places by replacing physical lines with on-demand virtual lines, had been recognized in early May as one of the top solutions in the Call for Code accelerated COVID-19 track. Since May, IBM specialists and partners have worked to further incubate, test, and deploy Safe Queue's solution with organizations across the country. "All of the submissions in this year's global challenge clearly show the immense potential of technologies based on hybrid cloud, AI and open source to address critical issues like climate change, COVID-19 and more," said Bob Lord, Senior Vice President, Cognitive Applications and Ecosystems, IBM. "We know the developer community has the skills, desire and ingenuity to tackle the world's thorniest issues. What we're providing through Call for Code is a catalyst to galvanize that community to take on specific societal challenges, as well as the open source-powered products and technologies to bring their vision to reality. Through this powerful combination, brilliant ideas like Agrolly can be transformed into the scalable solutions the world needs today." Winners in the University Category Chelsea Clinton, Vice Chair, Clinton Foundation, announced the inaugural winner of the Call for Code University Edition, a collaboration between IBM and the Clinton Global Initiative University. Pandemap, created by a team of students from UC Berkeley to monitor and manage crowd flow and promote social distancing during COVID-19 will receive $10,000. Lupe, created by university students in the United Kingdom, was named runner-up. Team members from Pandemap and Lupe also receive the opportunity to interview for a potential role at IBM. "Now, more than ever, the scope and urgency of the issues we're encountering demand diverse perspectives and expertise and we're proud to partner with IBM for the second year to advance university efforts that are committed to doing just that," Clinton said. "Reaching over 53,000 students from more than 45 nations in 2020, we saw a tremendous and inspiring movement of young people, investing their time and talent during Call for Code. The passion, collaboration and innovation of our students is what will help unite and propel our society forward." Advancing Racial Justice The announcement of Call for Code for Racial Justice follows three years of successful global programs addressing natural disasters, climate change and COVID-19. Call for Code for Racial Justice encourages the adoption and innovation of open source projects to drive progress in three key areas of focus: Police & Judicial Reform and Accountability; Diverse Representation; and Policy & Legislation Reform. Together with partners like Black Girls Code, Collab Capital, Dream Corps, The United Way Worldwide, American Airlines, Cloud Native Computing Foundation and Red Hat, Call for Code for Racial Justice is inviting developers to apply their skills and ingenuity to combat systemic racism. The tragic deaths of George Floyd, Amaud Arbery, Breonna Taylor, and many others, serve as a reminder that silent carriers help spread racism, and the fight against it is as urgent as ever. The new initiative began with Black IBMers and allies taking action with an internal IBM program called the Call for Code Emb(race) Challenge. Solutions created and developed through that program are now being opened to the world to build upon through Call for Code for Racial Justice.   "Black Girls Code was created to introduce programming and technology to a new generation of coders, and we believe that a new generation of coders will shape our futures,'' said Anesha Grant, Director of Alumnae and Educational Programs, Black Girls Code. "We're excited to participate in Call for Code for Racial Justice and to spark meaningful change." Call for Code for Racial Justice is planned for launch at the virtual All Things Open on October 19. "Each year I'm amazed by how this global community of developers comes together to help solve some of the world's most pressing issues, and this year is no different," said Call for Code creator David Clark. "Through the support of UN Human Rights, IBM, The Linux Foundation, the Call for Code ecosystem, world leaders, tech icons, celebrities, and the amazing developers that drive innovation, Call for Code has become the defining tech for good platform the world turns to for results." About Call for Code Global Challenge Developers have revolutionized the way people live and interact with virtually everyone and everything. Where most people see challenges, developers see possibilities. That's why David Clark, the CEO of David Clark Cause, created Call for Code in 2018, and launched it alongside Founding Partner IBM and their partner UN Human rights. This five-year, $30 million global initiative is a rallying cry to developers to use their mastery of the latest technologies to drive positive and long-lasting change across the world through code. The Call for Code community includes United Nations Human Rights, The Linux Foundation, United Nations Office for Disaster Risk Reduction, Clinton Foundation and Clinton Global Initiative University, Cloud Native Computing Foundation, Verizon, Persistent Systems, Arrow Electronics, HERE Technologies, Ingram Micro, IntelePeer, Consumer Technology Association Foundation, World Bank, Caribbean Girls Hack, Kode With Klossy, World Institute on Disability, and many more. Call for Code global winning solutions are further developed, incubated, and deployed as sustainable open source projects to ensure they can drive positive change. Media Contact:Deirdre Leahy(845) 863-4552deirdre.leahy@ibm.com SOURCE IBM

IBM and The Climate Service to Work Together with Financial Institutions and Corporations to Assess the Cost of Climate Risk

Collaboration to Provide Organizations an IBM Cloud-based Platform to Measure, Quantify and Disclose Climate Risk ARMONK, N.Y. and DURHAM, N.C., Oct. 14, 2020 /PRNewswire/ -- IBM (NYSE: IBM) and The Climate Service (TCS), a leader in climate risk analytics for investors and businesses, today announced their alliance to work with financial institutions and corporations to better measure and quantify risks associated with climate change. As part of the alliance, the companies are now making the TCS Climanomics® software platform available via Red Hat OpenShift on IBM Cloud. This alliance between IBM and TCS makes TCS's advanced science available with enterprise-grade levels of scalability, availability, security, and interoperability. The companies are working together to help organizations put a price on climate risk and facilitate reporting consistent with the Task Force on Climate-Related Financial Disclosures (TCFD) framework. The alliance will also leverage the experience of IBM Services in financial risk, analytics, weather risk, and compliance, to use the Climanomics® platform to provide climate risk analytics to clients across all sectors, including global financial services leaders, industrial corporations, and energy providers. "At a time when markets and investors are demanding climate risk transparency and quantification, IBM is committed to providing cloud-based climate risk solutions to the financial services sector and critical infrastructure industries such as industrial, consumer, energy, and utilities," said Dr. Murray Simpson, IBM's Global Lead for Sustainability, Climate & Transition. "This collaboration with The Climate Service will provide clients financial insights that are based on the most advanced available science and technology. As we work with clients to integrate critical insights into their business decision-making, our goal is to inform their strategy and help them remain competitive and resilient." "With the TCFD framework now supported by organizations globally, demand for the Climanomics® platform has been growing fast," says James McMahon, CEO of The Climate Service. "With IBM's advanced technology and global experience, this work will scale quickly so we can accelerate progress on our mission to embed climate risk into global decision-making." "Climate change presents the markets, as well as society as a whole, with an exponential challenge. Through this collaboration we will offer solutions using the power of world-class science and technology," continued McMahon. IBM and TCS signed their alliance agreement in October 2019, and the two companies have been collaborating for the past year, building a go-to-market strategy. This announcement is made on the heels of the collaboration reaching several milestones that enabled the TCS Climanomics® software platform to be available on Red Hat OpenShift on IBM Cloud, as well as the commencement of a joint effort to find ways to help global financial and energy companies measure climate risk. About The Climate ServiceThe Climate Service provides climate risk analytics to investors and businesses. With support from an unparalleled scientific team and advisors including four IPCC Nobel Prize-winning scientists, The Climate Service built its Climanomics® analytics platform to enable reporting and disclosure consistent with the TCFD framework. The platform empowers decision-making by putting a price on both transition and physical risk and supports long-term, strategic planning with 'one-click' property/asset and portfolio-level analysis, and multiple climate scenarios covering present-day to the year 2100. Insights from the platform help investors and businesses manage risk, build resilience, and drive change. Forrester Research named The Climate Service as "Leader", the highest designation possible, in its Climate Risk Analytics New Wave 2020. Contacts:Chris AndrewsIBM Communicationscandrews@us.ibm.com  Julianne HoganJulianne@benecomms.io Disclaimer This document is current as of the initial date of publication and may be changed by IBM at any time. Not all offerings are available in every country in which IBM operates.It is the user's responsibility to evaluate and verify the operation of any other products or programs with IBM products and programs. IBM products are warranted according to the terms and conditions of the agreements under which they are provided. SOURCE IBM