Google high-level FAQ around Generative AI
What is a transformer ?
Is a type of neural network architecture which implements an innovative mechanism to learn patterns called self-attention. It’s technically complex to explain, but essentially self-attention allows algorithms to output data based on the attention provided in parts of the input data, from different perspectives (why, when, how, …).
A transformer has two key advantages: 1. Can be fine-tuned to multiple natural language tasks (summarization, conversations, QA, …) 2. Allows fast parallelization with TPUs, since self-attention mechanisms use a lot of matrix multiplications, which TPUs do very well.
Google invented the transformers back in 2017, and has released it at scale in Google products (for example, BERT in Google search in 2018). Google has algorithms like PaLM, LaMDA and Sparrow (DeepMind). Google has an even bigger list of transformers-based algorithms, like MUM, LaMDA, Bard, mT5, UL2, Pegasus, BERT and many others. Some few are public and open source, some others don’t.
TPUs and transformers are a perfect match. You can train and fine-tune transformer-based models faster and cheaper with TPUs.
How Google deals with Responsible AI ?
There is a public blog article, signed by Sundar Pichai, Demis Hassabis, James Manyika and Jeff Dean (“Why we focus on AI ? (and to what end)”) showing Google perspective on the potential of AI. Key quotes:
- “We believe that AI is a foundational and transformational technology that will provide compelling and helpful benefits to people and society through its capacity to assist, complement, empower, and inspire people in almost every field of human endeavor.”
- “At the same time, we understand that AI, as a still-emerging technology, poses various and evolving complexities and risks. Our development and use of AI must address these risks. That’s why we as a company consider it an imperative to pursue AI responsibly.”
- “We also believe that getting AI right — which to us involves innovating and delivering widely accessible benefits to people and society, while mitigating its risks — must be a collective effort involving us and others, including researchers, developers, users (individuals, businesses, and other organizations), governments, regulators and citizens.” To summarize,(quoted from article): “as leaders in AI, we must lead not only in state-of-the-art AI technologies, but also in state-of-the-art responsible AI. We are excited about what lies ahead in 2023 and beyond as we get ready to share some new innovative experiences” While there are no Google public launches like ChatGPT (LaMDA, PaLM), note DeepMind has announced a private beta of Sparrow during 2023. Sparrow provides even some advantages over ChatGPT, like no invented information, or citing sources.
Generative AI products for consumers
The following is a list of GenAI products targeted at consumer space and out of the SLAs provided by Google Cloud for enterprise customers:
- Bard: experimental conversational AI service.
- MakerSuite: tool that simplifies the deployment of LLMs. With MakerSuite, you’ll be able to iterate on prompts, augment your dataset with synthetic data, and easily tune custom models
- NotebookLM (formerly project Tailwind): experiment for notetaking.
References
[1] Google blog: Why we focus on AI ? (and to what end)
[2] Google blog: An important next step on our AI journey
[3] Google blog: The next generation of AI for developers and Google Workspace
[4] Google Cloud blog: Google Cloud brings generative AI to developers, businesses, and governments
[5] YouTube video: Build, tune and deploy foundation models with vertex AI
[6] Google developers blog: PaLM API & MakerSuite: an approachable way to start prototyping and building generative AI applications
[7] Google Cloud blog: NL APIs based on transformers for text classification
[8] Github repo: Train and fine-tune transformers with Vertex AI
[9] Github repo: Finetune advanced algorithms like T5 on Vertex AI and Cloud TPUs
[10] Github repo: Finetune advanced algorithms like Flan-T5 on Vertex AI