Firmware
Tesla AI Director explains how vision-based strategies can make autonomy mainstream
It’s almost amusing, but rarely is Tesla’s Autopilot data considered when Wall Street analysts are evaluating the company. Even among its ardent bulls, very few apart from ARK Invest CEO Cathie Wood seems to take the company’s efforts at attaining full self-driving seriously in their forecasts. Part of this may be because autonomous driving is not really here yet. The regulations are not ready, and the technology is still catching up to the hype.
Tesla is dedicated to pursuing full self-driving using an approach that is heavily reliant on vision. Using its electric cars’ suite of cameras and artificial intelligence, Tesla can train its neural networks in how it responds to circumstances on the road. Other mainstream autonomous driving firms do not adopt this approach. Both Waymo and Cruise, both of whom are considered as the leaders in the field, rely on LiDAR, which Tesla CEO Elon Musk describes as a fool’s errand.
This approach was explained by Tesla AI Director Andrej Karpathy in a lecture last February, where he discussed the electric car maker’s strategies for its autonomous driving program. Karpathy covered several topics, from the challenges involved in long tail events as well as the importance of a vision-based approach in the widespread adoption and rollout of full self-driving solutions.
Critics of Tesla’s Autopilot and Full Self-Driving strategies would be quick to state that the company’s demonstrations, such as a clip of an autonomous Model 3 shown following Autonomy Day, were unremarkable since other companies like Waymo have done similar demos for years. This, Karpathy explained, is not the case, since Tesla follows a far different approach to achieve autonomy. “The critical point to make is that it looks the same, but under the hood, it’s completely different in terms of the approach that we take,” he said.
Companies like Waymo use LiDARs and high-definition maps to navigate around an area that’s intricately pre-mapped. It’s a sound approach, though it is also pretty restrictive. Waymo, for example, has been developing its autonomous fleet for years, but so far, it could only operate in a few locations. This is ultimately the reason why the grand scheme of things, a LiDAR-based FSD approach may not be feasible. There are simply far too many uncertainties on the road, both in highways and in inner-city streets, that would likely make a pre-mapped solution ineffective.
Tesla does not use high-definition maps. Instead, the company primarily depends on its vehicles’ cameras and artificial intelligence to drive without human input. This is very similar to the way humans drive, since people utilize their eyes to watch the road, and they use their brain to decide what actions to take while driving. Karpathy noted that this approach provides an opportunity for Tesla to roll out its Autopilot and Full Self-Driving improvements to its entire fleet efficiently. This allows Tesla to roll out an improved version of its Autopilot software that’s not in any way restricted to pre-mapped data.
Doing this is a complicated process, of course. To say that neural network training is a tedious process is an understatement, especially when it comes to long tail cases. Fortunately, this is a challenge that Tesla is all-too-willing to take. Karpathy’s lecture ultimately highlighted the foresight that goes behind Tesla’s efforts to develop a full self-driving solution that can be used anywhere.
It’s a bit amusing to note the similarities to the reactions that Tesla’s FSD efforts are receiving today and the reactions that the Model S program received from naysayers years ago. When the Model S was announced, critics noted that it was a fool’s errand. Others dismissed it as vaporware. No one has really made a premium, modern, production electric vehicle from the ground up then, so it would have been easier for Tesla to stay as a niche automaker. But staying as a niche automaker of high-performance electric sports cars doesn’t align with the company’s mission at all.
For Tesla to help accelerate the transition to sustainability, it must have mass market vehicles, and the Model S was the first of that. Doing so was difficult and almost like an act of hubris for the company then, as represented by the purchase of the Fremont factory that was incredibly large for the Model S production line. But Tesla did it anyway, and years after, the company stands just next to Toyota in terms of market cap. Similar to how Tesla eventually proved everyone wrong with the Model S, the electric car maker may very well end up proving its critics wrong once more when it releases its full self-driving features to its fleet.
Watch Tesla AI Director Andej Karpathy’s lecture in the video below.
It appears that Tesla may be preparing to roll out some subscription-based services soon. Based on the observations of a Wales-based Model 3 owner who performed some reverse-engineering on the Tesla mobile app, it seems that the electric car maker has added a new “Subscribe” option beside the “Buy” option within the “Upgrades” tab, at least behind the scenes.
A screenshot of the new option was posted in the r/TeslaMotors subreddit, and while the Tesla owner in question, u/Callump01, admitted that the screenshot looks like something that could be easily fabricated, he did submit proof of his reverse-engineering to the community’s moderators. The moderators of the r/TeslaMotors subreddit confirmed the legitimacy of the Model 3 owner’s work, further suggesting that subscription options may indeed be coming to Tesla owners soon.
Did some reverse engineering on the app and Tesla looks to be preparing for subscriptions? from r/teslamotors
Tesla’s Full Self-Driving suite has been heavily speculated to be offered as a subscription option, similar to the company’s Premium Connectivity feature. And back in April, noted Tesla hacker @greentheonly stated that the company’s vehicles already had the source codes for a pay-as-you-go subscription model. The Tesla hacker suggested then that Tesla would likely release such a feature by the end of the year — something that Elon Musk also suggested in the first-quarter earnings call. “I think we will offer Full Self-Driving as a subscription service, but it will be probably towards the end of this year,” Musk stated.
While the signs for an upcoming FSD subscription option seem to be getting more and more prominent as the year approaches its final quarter, the details for such a feature are still quite slim. Pricing for FSD subscriptions, for example, have not been teased by Elon Musk yet, though he has stated on Twitter that purchasing the suite upfront would be more worth it in the long term. References to the feature in the vehicles’ source code, and now in the Tesla mobile app, also listed no references to pricing.
The idea of FSD subscriptions could prove quite popular among electric car owners, especially since it would allow budget-conscious customers to make the most out of the company’s driver-assist and self-driving systems without committing to the features’ full price. The current price of the Full Self-Driving suite is no joke, after all, being listed at $8,000 on top of a vehicle’s cost. By offering subscriptions to features like Navigate on Autopilot with automatic lane changes, owners could gain access to advanced functions only as they are needed.
Elon Musk, for his part, has explained that ultimately, he still believes that purchasing the Full Self-Driving suite outright provides the most value to customers, as it is an investment that would pay off in the future. “I should say, it will still make sense to buy FSD as an option as in our view, buying FSD is an investment in the future. And we are confident that it is an investment that will pay off to the consumer – to the benefit of the consumer.” Musk said.
Tesla has started rolling out update 2020.36 this weekend, introducing a couple of notable new features for its vehicles. While there are only a few handful of vehicles that have reportedly received the update so far, 2020.36 makes it evident that the electric car maker has made some strides in its efforts to refine its driver-assist systems for inner-city driving.
Tesla is currently hard at work developing key features for its Full Self-Driving suite, which should allow vehicles to navigate through inner-city streets without driver input. Tesla’s FSD suite is still a work in progress, though the company has released the initial iterations of key features such Traffic Light and Stop Sign Control, which was introduced last April. Similar to the first release of Navigate on Autopilot, however, the capabilities of Traffic Light and Stop Sign Control were pretty basic during their initial rollout.
2020.36 Showing Speed Limit Signs in Visualization from r/teslamotors
With the release of update 2020.36, Tesla has rolled out some improvements that should allow its vehicles to handle traffic lights better. What’s more, the update also includes a particularly useful feature that enables better recognition of speed limit signs, which should make Autopilot’s speed adjustments better during use. Following are the Release Notes for these two new features.
Green Traffic Light Chime
“A chime will play when the traffic light you are waiting for turns green. If you are waiting behind another car, the chime will play once the car advances unless Traffic-Aware Cruise Control or Autosteer is active. When Traffic Light and Stop Sign Control is activated, a chime will play when you can confirm to proceed through a green traffic light. To enable, tap Controls > Autopilot > Green Traffic Light Chime.
“Note: This chime is only designed as a notification. It is the driver’s responsibility to observe their environment and make decisions accordingly.”
Speed Assist Improvements
“Speed Assist now leverages your car’s cameras to detect speed limit signs to improve the accuracy of speed limit data on local roads. Detected speed limit signs will be displayed in the driving visualization and used to set the associated Speed Limit Warning.
“As usual, to adjust Speed Assist settings, tap Controls > Autopilot > Speed Limit.”
Footage of the new green light chime in action via @NASA8500 on Twitter ✈️ from r/teslamotors
Amidst the rollout of 2020.36’s new features, speculations were abounding among Tesla community members that this update may include the first pieces of the company’s highly-anticipated Autopilot rewrite. Inasmuch as the idea is exciting, however, Tesla CEO Elon Musk has stated that this was not the case. While responding to a Tesla owner who asked if the Autopilot rewrite is in “shadow mode” in 2020.36, Musk responded “Not yet.”
Tesla has rolled out a free three-month trial subscription to Sirius XM, in what appears to be the company’s latest push into making its vehicles’ entertainment systems more feature-rich. The new Sirius XM offer will likely be appreciated by owners of the company’s vehicles, especially considering that the service is among the most popular satellite radios in the country today.
Tesla announced its new offer in an email sent on Monday. An image that accompanied the communication also teased Tesla’s updated and optimized Sirius XM UI for its vehicles. Following is the email’s text.
“Beginning now, enjoy a free, All Access three-month trial subscription to Sirius XM, plus a completely new look and improved functionality. Our latest over-the-air software update includes significant improvements to overall Sirius XM navigation, organization, and search features, including access to more than 150 satellite channels.
“To access simply tap the Sirius XM app from the ‘Music’ section of your in-car center touchscreen—or enjoy your subscription online, on your phone, or at home on connected devices. If you can’t hear SiriusXM channels in your car, select the Sirius XM ‘Subscription’ tab for instruction on how to refresh your audio.”
Tesla has actually been working on Sirius XM improvements for some time now. Back in June, for example, Tesla rolled out its 2020.24.6.4 update, and it included some optimizations to its Model S and Model X’s Sirius XM interface. As noted by noted Tesla owner and hacker @greentheonly, the source code of this update revealed that the Sirius XM optimizations were also intended to be released to other areas such as Canada.
Interestingly enough, Sirius XM is a popular feature that has been exclusive to the Model S and X. Tesla’s most popular vehicle to date, the Model 3, is yet to receive the feature. One could only hope that Sirius XM integration to the Model 3 may eventually be included in the future. Such an update would most definitely be appreciated by the EV community, especially since some Model 3 owners have resorted to using their smartphones or third-party solutions to gain access to the satellite radio service.
The fact that Tesla seems to be pushing Sirius XM rather assertively to its customers seems to suggest that the company may be poised to roll out more entertainment-based apps in the coming months. Apps such as Sirius XM, Spotify, Netflix, and YouTube, may seem quite minor when compared to key functions like Autopilot, after all, but they do help round out the ownership experience of Tesla owners. In a way, Sirius XM does make sense for Tesla’s next-generation of vehicles, especially the Cybertruck and the Semi, both of which would likely be driven in areas that lack LTE connectivity.
Armored Tesla Cybertruck “War Machine” debuts at Defense Expo 2025
Tesla Megapacks chosen for 548 MWh energy storage project in Japan
