Hot Chips 2021 Keynote Live Blog: Skydio and Autonomous Flight

07:02PM EDT - Welcome to Hot Chips! This is the annual conference all about the latest, greatest, and upcoming big silicon that gets us all excited. Stay tuned during Monday and Tuesday for our regular AnandTech Live Blogs.

07:04PM EDT - Second keynote today is from Skydio

07:05PM EDT - Should be starting here in a second

07:06PM EDT - This talk is about autonomous flight

07:07PM EDT - 'Enabling next generation'

07:08PM EDT - I always think 'autonomous flight' is about flying people, but this is more about the next gen of drones

07:09PM EDT - Starting with the history of drones

07:09PM EDT - First ones were gas powered

07:09PM EDT - Then battery powered

07:10PM EDT - MEMs IMUs and GPUs

07:11PM EDT - Replacing legacy methods cheaper and safer and faster

07:11PM EDT - Cinematography, safety, inspection

07:11PM EDT - 80% of an average drone budget can be consumed by pilot training

07:12PM EDT - confined flying control with ML

07:13PM EDT - looks kinda cool

07:13PM EDT - Moving from age of manual hardware to AI-driven drones

07:14PM EDT - Liken the tradition similar to flip phone to smart phone

07:14PM EDT - now a video showing usecases

07:15PM EDT - First US drone unicorn - $340M funding

07:16PM EDT - World leader in autonomous flight technology

07:16PM EDT - $5.1B market in sales today

07:16PM EDT - 2.3m units

07:17PM EDT - When drones are the size of hummingbirds, what is the market reach?

07:18PM EDT - 100x enterprise growth over 10 years

07:18PM EDT - Today in stage 1/stage 2

07:19PM EDT - Always someone hand-holding the drone right now

07:20PM EDT - stage 3 is basically Drone as a Service - take photos of your kids sports game with an app

07:21PM EDT - Skydio autonomy engine

07:21PM EDT - Sensors, algorithms

07:21PM EDT - 6x4K 360-degree fisheye cameras

07:22PM EDT - That's more than a tesla?

07:22PM EDT - Understand the geometry of the world around it

07:23PM EDT - Depth mapping

07:23PM EDT - Problem is detecting thin structures and wires

07:24PM EDT - understanding the environment

07:24PM EDT - keeping someone in the shot regardless of terrain

07:25PM EDT - generating a 3D view of the world, 3D mapping live and evolving

07:27PM EDT - Simulated examples of use cases

07:27PM EDT - Building 3D maps in real time as detailed as needed

07:28PM EDT - 3D scan sounds cool

07:29PM EDT - Using NVIDA and Qualcomm

07:29PM EDT - Jetson TX2

07:30PM EDT - Qualcomm for communication I'd imagine, or Qualcomm for ISP

07:30PM EDT - Imaging SoC (QC) and Navigation SoC (NV)

07:31PM EDT - Power budget - 100-200W consumes, 20-40 minute flytime. 1W of compute reduces flight time by 10 seconds

07:31PM EDT - Compute constraint is mostly driven by size/weight of the board

07:32PM EDT - Types of compute

07:34PM EDT - Deep Learning inference used in flight - 8 models

07:35PM EDT - It's a big optimization problem

07:36PM EDT - model with predictive control

07:36PM EDT - Motion planning with 6DOF

07:36PM EDT - How the rotor rates should evolve over time

07:36PM EDT - 500 iterations a second with 20+ objectives

07:36PM EDT - 'smooth cinematic flight'

07:37PM EDT - Matrix solvers as per usual

07:39PM EDT - Custom differential geometry library

07:39PM EDT - void handwritten derivatives

07:39PM EDT - high level symbolic python code

07:39PM EDT - Generated code is branchless

07:40PM EDT - Provided an order of magnitude speed-up on CPU

07:40PM EDT - finding ways to apply to other hardware through standard interfaces

07:41PM EDT - Still very early

07:41PM EDT - Real world feedback - foundational building blocks

07:42PM EDT - needs to handle all situations, e.g. sun glare, in a way that's accessible but optimized

07:43PM EDT - Q&A time

07:45PM EDT - Q: Compare to DJI - talk about market is evolving A: key difference with Skydio is in the software and what the user is asking them to do. Industry has been smoke and mirrors - what really matters is true intelligence, need to perform the job and not just be pie in the sky - same with autonomous cars, easy to demo, hard to work 100% of the time or 99.99999%

07:46PM EDT - Q: Call to action - build better processors: Expand, what's the dream? A: Ton of opportunity. Getting the balance of power and functionality is important. Workloads need to be addressed with cameras, more than most, so need to process them with latency and power is key. Do that into a single ASIC is what we need. 2x the compute, but it's all about the size, and 10x would be nice

07:51PM EDT - Q: Sensor suite A: Grad school used LIDAR, today with sensors - LIDARs are orders of magnitude not applicable for the size of drones. Cameras are getting better, Computer vision is getting a lot better, compute is getting a lot better. Exploit it.

07:53PM EDT - Q: Simulated data to train models? How often to retain? A: Becomes very easy (relatively) when you see failure cases. Can mine and exploit data and work into the training set. That data is key. Balancing of data is important. Simulated data is useful, but need real data and bring it together to get to the level of performance you need

07:54PM EDT - Q: Portable algorithms? A: We do specific things that you tailor the software to do, like deep learning toolkits, tensorRT, you would need to reoptimize

07:58PM EDT - Q: Ideal battery? New Tech on the horizon? A: It's a key perf metric. Battery has been getting better slowly over time. Keep abreast over time. Needs aren't crazy - we care more about safety and capacity. Needs are well aligned with laptops and cars.

07:59PM EDT - Q: Hot swap mid-flight? A: Anything is possible. Challenging to do well relative to benefit. Drones could live in docking stations / replace batteries is a better solutions. If you need constant, just use multiple drones

08:02PM EDT - That's a wrap!