Google Cloud | What is Bigtable

How BIG Is Cloud Bigtable?

“Building an application that needs low latency and high throughput?—You need a database that can scale for a large number of reads and writes. Cloud Bigtable is designed to handle just that. 

Cloud Bigtable is a fully managed wide-column NoSQL database that scales to petabyte-scale. It’s optimized for low latency, large numbers of reads and writes, and maintaining performance at scale. It offers really low latency of the order of single-digit milliseconds. It is an ideal data source for time series and MapReduce-style operations. Bigtable supports the open-source HBase API standard to easily integrate with the Apache ecosystem including HBase, Beam, Hadoop and Spark. It also integrates with Google Cloud ecosystem including Memorystore, BigQuery, Dataproc, Dataflow and more. 

Google Cloud | What is Bigtable

Some Cloud Bigtable Features

  • Data is by default encrypted with Google managed encryption keys but, for specific compliance and regulatory requirements if customers need to manage their own keys, customer managed encryption keys (CMEK) are also supported.
  • Bigtable backups let you save a copy of a table’s schema and data, then restore from the backup to a new table at a later time. Backups can help you recover from application-level data corruption or from operator errors such as accidentally deleting a table.

Scale and High Availability (HA)

How BIG is Bigtable? Bigtable has nearly 10 Exabytes of data under management. 

It delivers highly predictable performance that is linearly scalable. Throughput can be adjusted by adding/removing nodes — each node provides up to 10,000 operations per second (read and write). You can use Bigtable as the storage engine for large-scale, low-latency applications as well as throughput-intensive data processing and analytics. It offers high availability with an SLA of 99.9% for zonal instances. It’s strongly consistent in a single cluster; replication between clusters adds eventual consistency. If you leverage Bigtable’s multi cluster routing across two clusters, the  SLA increases to 99.99% and if that routing policy is utilized across clusters in 3 different regions you get a 99.999% uptime SLA.

Replication for Cloud Bigtable enables you to increase the availability and durability of your data by copying it across multiple regions or multiple zones within the same region. To use replication in a Bigtable instance, just create an instance with more than 1 cluster or add clusters to an existing instance. Bigtable supports up to 4 replicated clusters located in Google Cloud zones where Bigtable is available. Placing clusters in different zones or regions enables you to access your data even if one zone or region becomes unavailable. Bigtable treats each cluster in your instance as a primary cluster, so you can perform reads and writes in each cluster. You can also set up your instance so that requests from different types of applications are routed to different clusters. The data and changes to data are synchronized automatically across clusters. 

How does it optimize throughput

Through separation of processing and storage, Cloud Bigtable is able to automatically configure throughput by adjusting the association of nodes and data. In the rebalancing example, if Node A is experiencing a heavy load, the routing layer can move some of the traffic to a less heavily loaded node, improving overall performance. Resizing comes into play when a node is added to again ensure a balanced load across nodes, ensuring best overall throughput.

Click to enlarge

Choice of app profile and traffic routing can also affect performance. An app profile with multi-cluster routing automatically routes requests to the closest cluster in an instance from the perspective of the application, and the writes are then replicated to the other clusters in the instance. This automatic choice of the shortest distance results in the lowest possible latency. An app profile that uses single-cluster routing can be optimal for certain use cases, like separating workloads or to have read-after-write semantics on a single cluster, but it will not reduce latency in the way multi-cluster routing does.

Replication can improve read throughput, especially when you use multi-cluster routing. And it can reduce read latency by placing your data geographically closer to your users.  Write throughput does not increase with replication because write to one cluster must be replicated to all other clusters in the instance. Resulting in each cluster spending the CPU resources to pull changes from the other clusters.

Conclusion

Bigtable is a database of choice for use cases that require a specific amount of  scale or throughput with strict latency requirements, such as IoT, AdTech, FinTech, gaming and ML based personalizations. You can ingest 100s of thousands of events per second from websites or IoT devices through Pub/Sub, process them in Dataflow and send them to Cloud Bigtable. For a more in-depth look into Cloud Bigtable check out the documentation or join the upcoming webinar with our experts Build high-throughput, low-latency apps with Cloud Bigtable.

For more #GCPSketchnote, follow the GitHub repo. For similar cloud content follow me on Twitter @pvergadia and keep an eye out on thecloudgirl.dev.

By: Priyanka Vergadia (Developer Advocate, Google)
Source: Google Cloud Blog



For enquiries, product placements, sponsorships, and collaborations, connect with us at hello@globalcloudplatforms.com. We'd love to hear from you!


Our humans need coffee too! Your support is highly appreciated, thank you!

Total
0
Shares
Previous Article
Google Cloud | Cloud Web Publishing | Clouds

Google Cloud VMware Engine Now HIPAA Compliant

Next Article
Google Cloud | AI ML

New Research Reveals What’s Needed For AI Acceleration In Manufacturing

Related Posts

Santa Casa da Misericórdia de Lisboa Delivering Telehealth Services on IBM Cloud

Patients can now schedule appointments and receive remote medical care through a mobile teleconsultation solution ARMONK, N.Y., Oct. 15, 2020 /PRNewswire/ -- IBM and Santa Casa da Misericórdia de Lisboa (SCML), a Portuguese institution of public utility recognized for its social, health and education work in the community, announced today they have worked together with IBM Garage and IBM Services to develop a teleconsultation solution running on IBM public cloud. The solution was designed to help people prioritize their health during the COVID-19 pandemic. IBM helped SCML develop a platform running on IBM public cloud that allows their patients to schedule and access teleconsultation services and remote medical care directly on their mobile phones. In addition to helping patients stay up-to-date with regular medical checkups, the platform is designed to reduce the need for patients to travel to a medical office or hospital and therefore, can help patients reduce exposure to COVID-19 when seeking medical care. The platform is designed so that patients can easily communicate with their medical providers and psychologists to get the assistance they need, quickly and from home. After the medical appointment is requested, the patient receives a notification by email or SMS with date and time information and the access link. Using a smartphone, the patient has to click on the link to start the teleconsultation. To date, SCML patients have used the platform to schedule nearly 300 medical appointments. For medical providers, the platform will enable them to initiate sessions through a phone call, video call or chat. Data and content only visible to the participants for the duration of the call, can build confidence for patients that their files are exchanged in a security rich, managed environment. Additionally, once the video call begins, it is kept between the doctor and patient without any third-party service or entity involved.   The IBM public cloud is the industry's most secure and open cloud for business, offering its  commercially-available state-of-art cryptographic technology for the cloud called "Keep Your Own Key."1 This gives enterprises the ability to retain control of their own encryption keys -- and the hardware security modules that protect them -- so customers are the only ones who can control access to their data. For clients in regulated industries like SCML, this need to protect client data is critical. Working together with a group of IBMers from the IBM Garage and IBM Services teams, SCML was able to develop this teleconsultation solution specifically tailored to its needs. As part of IBM's commitment to help organizations tap into the power of our many services and applications so they can maintain productivity in remote work environments during COVID, SCML was able to access a set of IBM services via the IBM public cloud at no cost for three months. Since most of SCML's patients will access the platform from a smartphone, the first adaption was to move from a computer-based telehealth solution that needed an email address for ID validation, to a solution that could be used on a mobile phone through SMS, audio calls and/or video calls. Running on IBM public cloud, the solution benefits from various features and functionalities supported by a robust containerized platform, able to scale according to demand. Additional foundational services were used for solution monitoring, access logging and user authentication. This solution assumes even more relevance as traditional methods have to be adapted as the world fights against COVID-19.  Portugal continues to face hundreds of new confirmed cases per day in the fall of 2020, mainly in the Metropolitan Area of Lisboa affirming this importance. Francisco Pessoa e Costa, Digital Transformation Manager of Santa Casa da Misericórdia de Lisboa adds, "We expect that this project will have great impact providing health care services in this pandemic period, especially among elderly, vulnerable and disadvantaged populations that we serve. In the middle of the pandemic situation in which we are living and all the instability and uncertainty that it brings, we are trying to do the best we can to protect those who need us. The entire SCML team is very proud and deeply involved in making this project a success and expanding its use among our users." "Challenges such as reimagining traditional operations during the COVID-19 pandemic give us an opportunity to apply cloud technology to make a real difference where it matters the most," said Gonçalo Costa Andrade, Enterprise Sales Manager, IBM Portugal. "This platform we helped build with SCML will enable its users to securely and conveniently access important medical, nutrition and psychology services." [1] Based on IBM Hyper Protect Crypto Service, the only service in the industry built on FIPS 140-2 Level 4-certified hardware. FIPS 140-2 Security Level 4 provides the highest level of security defined in this standard. At this security level, the physical security mechanisms provide a comprehensive envelope of protection around the cryptographic module with the intent of detecting and responding to all unauthorized attempts at physical access. About SCML SCML has more than five centuries of existence and a strong role in the Social Area. It's also developing important work in the Health, Education and Training, Culture, Research and Innovation areas, as well as in the area of Social Economy of Entrepreneurship. SCML guides its actions with the community by defending fundamental social values, in a perspective of proximity, respect and advocacy for good causes. About IBM Cloud For further information visit: www.ibm.com/cloud/ CONTACT:Kate GazzilloIBM Communicationskate.gazzillo@ibm.com  SOURCE IBM
Read More

Call for Code Names Winner of 2020 Global Challenge and Announces New Initiative to Combat Racial Injustice

Agrolly selected for innovative solution to help small farmers as they struggle with effects of climate changeNew initiative enables global developers to leverage proven Call for Code platform to help combat systemic racism ARMONK, N.Y., Oct. 13, 2020 /PRNewswire/ -- Where most people see challenges, developers see possibilities. That's certainly been the case in 2020. Today, Call for Code founding partner IBM (NYSE: IBM) and its creator, David Clark Cause, announced the winner of the 2020 Call for Code Global Challenge. The top prize went to Agrolly, an application to help the world's small farmers cope with the environmental and business challenges of climate change.   Call for Code also introduced a new initiative—Call for Code for Racial Justice—to urge its international community of hundreds of thousands of developers to contribute to solutions to confront racial inequalities. The announcements came during a virtual event, the "2020 Call for Code Awards: A Global Celebration of Tech for Good." A full replay can be watched here. The 2020 Call for Code Global Challenge had asked developers to create solutions to help communities fight back against climate change and COVID-19. A panel of industry leaders and judges awarded Agrolly the grand prize while announcing four other winners—one that also created a response to climate change, and three others aimed at the global coronavirus pandemic. Agrolly will receive $200,000, support from IBM Service Corps and technical experts, and ecosystem partners to incubate, test and deploy their solution. Agrolly will also receive assistance from The Linux Foundation to open-source their application so developers across the world can improve and scale the technology. Call for Code unites hundreds of thousands of developers to create and deploy applications powered by open source technology that can tackle some of the world's biggest challenges. Since its launch in 2018, this movement has grown to more than 400,000 developers and problem solvers across 179 nations, and has generated more than fifteen thousand solutions using technology including Red Hat OpenShift, IBM Cloud, IBM Watson, IBM Blockchain, data from The Weather Company, and APIs from ecosystem partners like HERE Technologies and IntelePeer. Top Solutions Tackling Effects of Climate Change and COVID-19 Agrolly was created by a distributed team of developers from Brazil, India, Mongolia and Taiwan who met at Pace University in New York City. Powered by IBM Cloud Object Storage, IBM Watson Studio, and IBM Watson Assistant, Agrolly aims to fill information gaps so farmers with limited resources can make more informed decisions, and obtain the necessary financing to help improve their economic outcomes. By combining weather forecasts from The Weather Company and historical data from NASA with crop requirements published by the United Nations Food and Agriculture Organization, Agrolly's platform provides tailored information for each farmer by location, crop type and even the plants' stage of development during the growing season. The Agrolly team, as part of their response to the Call for Code Challenge, has made the solution available as an app in the Google store, free of charge. Another climate change solution, OffShip, received fifth place and was awarded $10,000. Three COVID-19 solutions were also honored. Second place went to Business Buddy, which will receive $25,000. Safe Queue was given third place and $25,000; SchoolListIt was awarded fourth place and $10,000. Safe Queue, an app that enables a safer way to manage entry during COVID-19 at shopping centers, small businesses, and polling places by replacing physical lines with on-demand virtual lines, had been recognized in early May as one of the top solutions in the Call for Code accelerated COVID-19 track. Since May, IBM specialists and partners have worked to further incubate, test, and deploy Safe Queue's solution with organizations across the country. "All of the submissions in this year's global challenge clearly show the immense potential of technologies based on hybrid cloud, AI and open source to address critical issues like climate change, COVID-19 and more," said Bob Lord, Senior Vice President, Cognitive Applications and Ecosystems, IBM. "We know the developer community has the skills, desire and ingenuity to tackle the world's thorniest issues. What we're providing through Call for Code is a catalyst to galvanize that community to take on specific societal challenges, as well as the open source-powered products and technologies to bring their vision to reality. Through this powerful combination, brilliant ideas like Agrolly can be transformed into the scalable solutions the world needs today." Winners in the University Category Chelsea Clinton, Vice Chair, Clinton Foundation, announced the inaugural winner of the Call for Code University Edition, a collaboration between IBM and the Clinton Global Initiative University. Pandemap, created by a team of students from UC Berkeley to monitor and manage crowd flow and promote social distancing during COVID-19 will receive $10,000. Lupe, created by university students in the United Kingdom, was named runner-up. Team members from Pandemap and Lupe also receive the opportunity to interview for a potential role at IBM. "Now, more than ever, the scope and urgency of the issues we're encountering demand diverse perspectives and expertise and we're proud to partner with IBM for the second year to advance university efforts that are committed to doing just that," Clinton said. "Reaching over 53,000 students from more than 45 nations in 2020, we saw a tremendous and inspiring movement of young people, investing their time and talent during Call for Code. The passion, collaboration and innovation of our students is what will help unite and propel our society forward." Advancing Racial Justice The announcement of Call for Code for Racial Justice follows three years of successful global programs addressing natural disasters, climate change and COVID-19. Call for Code for Racial Justice encourages the adoption and innovation of open source projects to drive progress in three key areas of focus: Police & Judicial Reform and Accountability; Diverse Representation; and Policy & Legislation Reform. Together with partners like Black Girls Code, Collab Capital, Dream Corps, The United Way Worldwide, American Airlines, Cloud Native Computing Foundation and Red Hat, Call for Code for Racial Justice is inviting developers to apply their skills and ingenuity to combat systemic racism. The tragic deaths of George Floyd, Amaud Arbery, Breonna Taylor, and many others, serve as a reminder that silent carriers help spread racism, and the fight against it is as urgent as ever. The new initiative began with Black IBMers and allies taking action with an internal IBM program called the Call for Code Emb(race) Challenge. Solutions created and developed through that program are now being opened to the world to build upon through Call for Code for Racial Justice.   "Black Girls Code was created to introduce programming and technology to a new generation of coders, and we believe that a new generation of coders will shape our futures,'' said Anesha Grant, Director of Alumnae and Educational Programs, Black Girls Code. "We're excited to participate in Call for Code for Racial Justice and to spark meaningful change." Call for Code for Racial Justice is planned for launch at the virtual All Things Open on October 19. "Each year I'm amazed by how this global community of developers comes together to help solve some of the world's most pressing issues, and this year is no different," said Call for Code creator David Clark. "Through the support of UN Human Rights, IBM, The Linux Foundation, the Call for Code ecosystem, world leaders, tech icons, celebrities, and the amazing developers that drive innovation, Call for Code has become the defining tech for good platform the world turns to for results." About Call for Code Global Challenge Developers have revolutionized the way people live and interact with virtually everyone and everything. Where most people see challenges, developers see possibilities. That's why David Clark, the CEO of David Clark Cause, created Call for Code in 2018, and launched it alongside Founding Partner IBM and their partner UN Human rights. This five-year, $30 million global initiative is a rallying cry to developers to use their mastery of the latest technologies to drive positive and long-lasting change across the world through code. The Call for Code community includes United Nations Human Rights, The Linux Foundation, United Nations Office for Disaster Risk Reduction, Clinton Foundation and Clinton Global Initiative University, Cloud Native Computing Foundation, Verizon, Persistent Systems, Arrow Electronics, HERE Technologies, Ingram Micro, IntelePeer, Consumer Technology Association Foundation, World Bank, Caribbean Girls Hack, Kode With Klossy, World Institute on Disability, and many more. Call for Code global winning solutions are further developed, incubated, and deployed as sustainable open source projects to ensure they can drive positive change. Media Contact:Deirdre Leahy(845) 863-4552deirdre.leahy@ibm.com SOURCE IBM