- The Rundown AI
- Posts
- Meta's video AI breakthrough
Meta's video AI breakthrough
PLUS: How to automate emails from spreadsheets
Rowan Cheung
July 30, 2024
Sign Up | Advertise | Tools | AI University
Welcome, AI enthusiasts.
Meta's new AI can identify and track any object in a video with just a few clicks, making Hollywood-level cutouts and effects accessible to everyone.
Could this open-source breakthrough be the key to unlocking a new era of AI-powered video editing? Let's get into it...
In today’s AI rundown:
Meta unveils SAM 2 for video AI
Zuck and Huang envision AI’s future
How to automate emails from spreadsheet data
Runway releases image-to-video AI
5 new AI tools & 4 new AI jobs
More AI & tech news
Read time: 4 minutes
LATEST DEVELOPMENTS
META
Image source: Meta
The Rundown: Meta just introduced Segment Anything Model 2 (SAM 2), an advanced AI model that can identify and track objects across video frames in real-time, marking a significant leap in video AI.
The details:
SAM 2 extends Meta's previous image segmentation capabilities to video, addressing challenges like fast movement and object occlusion.
The model can segment any object in a video and create cutouts in a few clicks — with a free demo available to try here.
Meta is also open-sourcing the model and releasing a large, annotated database of 50,000 videos used for training.
Potential applications include video editing, mixed reality experiences, and scientific research.
Why it matters: SAM 2’s ability to track objects in real-time could make complex video editing tasks like object removal or replacement as simple as a single click. With Llama 3.1 last week and now SAM 2, Meta is continuing its strategy of developing massive AI breakthroughs — while making everything open and free to use.
TOGETHER WITH OCTOAI
The Rundown: OctoAI is delivering a deep dive into optimizing small, open-source models like Llama3.1-8B to overpower the big closed source providers — all while controlling your own data and slashing costs.
Join the technical session on Aug. 7 to see:
An interactive path to optimal model performance
Efficient approaches to fine tune the new Llama 3.1-8B model family
The continuous development cycle for LLM-powered apps
+47% quality improvements and +90% cost reduction over GPT-4o
Sign up for the session here. All registrants will receive the post-session materials, even if you can’t attend live!
SIGGRAPH 2024
Image source: NVIDIA
The Rundown: During a fireside chat at SIGGRAPH 2024, Meta CEO Mark Zuckerberg and NVIDIA CEO Jensen Huang spoke about their shared vision for the AI-powered future.
Key highlights:
Both CEOs emphasized the importance of open-source AI, with Zuckerberg highlighting Llama 3.1's release as an "inflection point."
Zuckerberg outlined a possible future for social media to evolve from recommending content to AI generating personalized content on the fly.
Huang predicted a shift from turn-based AI interactions to more fluid, multi-option simulations.
The leaders also discussed AI's potential to transform education, entertainment, and work through smart glasses.
Why it matters: The emphasis on open-source and personalized AI signals a potential shift in how AI will be integrated into everyday life and business. With Meta and NVIDIA's combined influence, the shared vision could significantly shape the future of AI and its applications across different industries.
AI TRAINING
The Rundown: Zapier Central allows you to automatically create emails from spreadsheet data, which can help you save time on repetitive tasks and ultimately boost your productivity.
Step-by-step:
Log in or sign up at Zapier and go to Zapier Central.
Create a new AI assistant named "Spreadsheet to Email".
Connect Google Sheets as your Data Source and set up the "Create Draft" Action for Gmail.
Test by instructing the AI to create a draft, then refine it as needed.
Pro tip: Start with email drafts instead of sending emails directly to allow you to review them and adjust if necessary before scaling up the automation!
PRESENTED BY APPYPIE
The Rundown: Appy Pie Playground is an all-in-one AI platform that revolutionizes creativity with its suite of AI tools, allowing users to effortlessly generate stunning videos, animations, and more.
Popular features include:
AI Video Generator: Create character-consistent videos with music & voiceovers
Animated Story Generator: Create animations with advanced story diffusion
Background Generator: Easily generate backgrounds for products and images
Cards Generator: Design and generate personalized cards
Get started today to generate your first AI video!
RUNWAY
Image source: Ideogram / The Rundown
The Rundown: Runway just announced that Gen-3 Alpha, the startup’s popular AI text-to-video generation model, can now create high-quality videos from still images.
The details:
According to Runway, image-to-video greatly improves the artistic control and consistency of video generations.
Image-to-video generations are either 5 or 10 seconds in length and take up “credits,“ which you have to pay for through Runway’s subscription tiers.
To use the tool, head to Runway’s website, click “try Gen-3 Alpha”, and upload an image to watch it come to life.
Why it matters: The highly anticipated image-to-video generation model opens up a whole new suite of creativity, allowing users to bring any image to life. However, while the increased artistic control and improvements to consistency are notable, Gen-3 Alpha does not come at a cheap price tag.
P.S. our favorite use case: Turning memes to life.
NEW TOOLS & JOBS
🎨 EverArt - Train AI on any product, style, or mood board
📝 Granola - AI-powered meeting note enhancement and organization
🎬 Runway Gen-3 Image to Video - Generate videos using images as guidance
🧑💻 Slater - Custom AI coding tool for Webflow website projects
🚀 fastn - Integrate and connect multiple data sources in a single, unified API
💻 Robust Intelligence - Senior Software Engineer (Cloud/Security)
👨💻 Palantir - Software Engineer, New Grad
🛠️ Meta - Product Manager - Infrastructure AI
📈 Databricks - Senior Customer Marketing Manager - EMEA
QUICK HITS
20% off AI Crash Course: Master prompting, AI productivity hacks, and proven use cases in Section’s 1-week AI Crash Course. 20% off with code RUNDOWN this week (Sponsored)
Meta launched AI Studio, a platform allowing users to create, share, and discover custom AI characters without technical skills, built on Llama 3.1.
Amazon unveiled its latest AI chip, claiming performance up to 50% higher than NVIDIA's at half the cost, as part of efforts to reduce reliance on external chips for its AWS AI cloud services.
Apple revealed its Apple Intelligence Foundation Model technical report, revealing architecture specifics, pre-training methods, and deployment strategies for both server and on-device AI models.
Scale AI announced its latest SEAL Leaderboard on Adversarial Robustness, evaluating AI models on their ability to handle red team-generated prompts across harm scenarios — with Google’s Gemini 1.5 Pro topping the leaderboard.
Topology introduced the Continuous Learning Model (CLM), a new AI system that can remember interactions, learn autonomously, and develop an ‘emergent’ personality.
Elon Musk predicted a future with 10 billion humanoid robots, more than the number of humans, describing it as a fundamental milestone in civilization.
THAT’S A WRAP
SPONSOR US
Get your product in front of over 600k+ AI enthusiasts
Our newsletter is read by thousands of tech professionals, investors, engineers, managers, and business owners around the world. Get in touch today.
FEEDBACK
How would you rate today's newsletter?Vote below to help us improve the newsletter for you. |
If you have specific feedback or anything interesting you’d like to share, please let us know by replying to this email.
Reply