Firmware
Tesla’s controversial vision-based full self-driving approach is finally paying off
Like many things about Elon Musk, Tesla’s approach to achieving autonomous driving is polarizing. Bucking the map-based trend set by industry veterans such as Waymo, Tesla opted to dedicate its resources in pursuing a vision-based approach to achieve full self-driving instead. This involves a lot of hard, tedious work on Tesla’s part, but today, there are indications that the company’s controversial strategy is finally paying off.
In a recent talk, Tesla AI Director Andrej Karpathy discussed the key differences between the map-based approach of Waymo and Tesla’s camera-based strategy. According to Karpathy, Waymo’s use of pre-mapped data and LiDAR make scaling difficult, since vehicles’ autonomous capabilities are practically tied to a geofenced area. Tesla’s vision-based approach, which uses cameras and artificial intelligence, is not. This means that Autopilot and FSD improvements can be rolled out to the fleet, and they would function anywhere.
This rather ambitious plan for Tesla’s full self-driving system has caught a lot of skepticism in the past, with critics pointing out that map-based FSD is the way to go. Tesla, in response, dug its heels in and doubled down on its vision-based initiative. This, in a way, resulted in Autopilot improvements and the rollout of FSD features taking a lot of time, particularly since training the neural networks, which recognize objects and driving behavior on the road, requires massive amounts of real-world data.
Perhaps this is the reason why Elon Musk’s estimates for a “feature complete” version of Full Self-Driving has been moved to this year, or why Smart Summon was delayed for months before its release. But inasmuch as this seemed to be the case, recent software updates and features from Tesla suggest that the pace of improvement for Autopilot and Full Self-Driving features seems to be accelerating.
Take Traffic Light and Stop Sign Control, for example. The feature made its debut this April, and despite travel still being somewhat restricted due to the pandemic, Tesla was able to get enough data from its fleet that the company is now looking to remove green light confirmation for the feature. The period between the feature’s initial release and its upcoming update lasted less than two months, which is significant considering that it is related to inner-city driving, one of the more challenging aspects of FSD.
Part of this may be due to the fact that Tesla’s fleet is simply far larger now. With Model 3 production in full swing in both the United States and in China, and with the Model Y now ramping, Tesla’s high-volume vehicles are getting to more and more customers. Each of these cars helps gather training data for the company’s neural network, which, in turn, aids in refining features like Traffic Light and Stop Sign Control. What’s interesting to note is that Tesla’s fleet is not even gathering data at its full potential, since several areas across the globe are still recovering from the effects of the coronavirus.
Once Tesla rolls out a version of Traffic Light and Stop Sign Control without green light confirmation, the company would pretty much be just one step away from having an FSD suite that’s feature-complete. The company’s US vehicle configurator shows this, with the FSD page only listing “Autosteer on city streets” as the lone feature of Full Self-Driving that is still dubbed “upcoming.” Considering the pace of improvement being rolled out to Autopilot and FSD improvements, it would not be surprising if Autosteer on city streets gets released soon as well.
It should be noted that every Autopilot improvement and FSD feature is the result of an insane amount of neural network training. Neural nets require mass amounts of data to learn driving behavior well, and so far, it appears that Tesla has now reached a point where its fleet could provide ample training data at a relatively quick timeframe. Tesla seems to be pretty optimistic about this too, as hinted at by the company’s recent update, which included the option to activate the Model 3 and Model Y’s cabin camera.
Elon Musk has stated in the past that the Model 3 and Model Y’s cabin camera will be used primarily when the Robotaxi Network goes online. Together with the quick improvements of features like Traffic Light and Stop Sign Control, it definitely appears that Tesla’s data-intensive, controversial, vision-based approach to full self-driving is finally coming together. And just like everything that the company has achieved so far, it was the result of very hard, very tedious work.
It appears that Tesla may be preparing to roll out some subscription-based services soon. Based on the observations of a Wales-based Model 3 owner who performed some reverse-engineering on the Tesla mobile app, it seems that the electric car maker has added a new “Subscribe” option beside the “Buy” option within the “Upgrades” tab, at least behind the scenes.
A screenshot of the new option was posted in the r/TeslaMotors subreddit, and while the Tesla owner in question, u/Callump01, admitted that the screenshot looks like something that could be easily fabricated, he did submit proof of his reverse-engineering to the community’s moderators. The moderators of the r/TeslaMotors subreddit confirmed the legitimacy of the Model 3 owner’s work, further suggesting that subscription options may indeed be coming to Tesla owners soon.
Did some reverse engineering on the app and Tesla looks to be preparing for subscriptions? from r/teslamotors
Tesla’s Full Self-Driving suite has been heavily speculated to be offered as a subscription option, similar to the company’s Premium Connectivity feature. And back in April, noted Tesla hacker @greentheonly stated that the company’s vehicles already had the source codes for a pay-as-you-go subscription model. The Tesla hacker suggested then that Tesla would likely release such a feature by the end of the year — something that Elon Musk also suggested in the first-quarter earnings call. “I think we will offer Full Self-Driving as a subscription service, but it will be probably towards the end of this year,” Musk stated.
While the signs for an upcoming FSD subscription option seem to be getting more and more prominent as the year approaches its final quarter, the details for such a feature are still quite slim. Pricing for FSD subscriptions, for example, have not been teased by Elon Musk yet, though he has stated on Twitter that purchasing the suite upfront would be more worth it in the long term. References to the feature in the vehicles’ source code, and now in the Tesla mobile app, also listed no references to pricing.
The idea of FSD subscriptions could prove quite popular among electric car owners, especially since it would allow budget-conscious customers to make the most out of the company’s driver-assist and self-driving systems without committing to the features’ full price. The current price of the Full Self-Driving suite is no joke, after all, being listed at $8,000 on top of a vehicle’s cost. By offering subscriptions to features like Navigate on Autopilot with automatic lane changes, owners could gain access to advanced functions only as they are needed.
Elon Musk, for his part, has explained that ultimately, he still believes that purchasing the Full Self-Driving suite outright provides the most value to customers, as it is an investment that would pay off in the future. “I should say, it will still make sense to buy FSD as an option as in our view, buying FSD is an investment in the future. And we are confident that it is an investment that will pay off to the consumer – to the benefit of the consumer.” Musk said.
Tesla has started rolling out update 2020.36 this weekend, introducing a couple of notable new features for its vehicles. While there are only a few handful of vehicles that have reportedly received the update so far, 2020.36 makes it evident that the electric car maker has made some strides in its efforts to refine its driver-assist systems for inner-city driving.
Tesla is currently hard at work developing key features for its Full Self-Driving suite, which should allow vehicles to navigate through inner-city streets without driver input. Tesla’s FSD suite is still a work in progress, though the company has released the initial iterations of key features such Traffic Light and Stop Sign Control, which was introduced last April. Similar to the first release of Navigate on Autopilot, however, the capabilities of Traffic Light and Stop Sign Control were pretty basic during their initial rollout.
2020.36 Showing Speed Limit Signs in Visualization from r/teslamotors
With the release of update 2020.36, Tesla has rolled out some improvements that should allow its vehicles to handle traffic lights better. What’s more, the update also includes a particularly useful feature that enables better recognition of speed limit signs, which should make Autopilot’s speed adjustments better during use. Following are the Release Notes for these two new features.
Green Traffic Light Chime
“A chime will play when the traffic light you are waiting for turns green. If you are waiting behind another car, the chime will play once the car advances unless Traffic-Aware Cruise Control or Autosteer is active. When Traffic Light and Stop Sign Control is activated, a chime will play when you can confirm to proceed through a green traffic light. To enable, tap Controls > Autopilot > Green Traffic Light Chime.
“Note: This chime is only designed as a notification. It is the driver’s responsibility to observe their environment and make decisions accordingly.”
Speed Assist Improvements
“Speed Assist now leverages your car’s cameras to detect speed limit signs to improve the accuracy of speed limit data on local roads. Detected speed limit signs will be displayed in the driving visualization and used to set the associated Speed Limit Warning.
“As usual, to adjust Speed Assist settings, tap Controls > Autopilot > Speed Limit.”
Footage of the new green light chime in action via @NASA8500 on Twitter ✈️ from r/teslamotors
Amidst the rollout of 2020.36’s new features, speculations were abounding among Tesla community members that this update may include the first pieces of the company’s highly-anticipated Autopilot rewrite. Inasmuch as the idea is exciting, however, Tesla CEO Elon Musk has stated that this was not the case. While responding to a Tesla owner who asked if the Autopilot rewrite is in “shadow mode” in 2020.36, Musk responded “Not yet.”
Tesla has rolled out a free three-month trial subscription to Sirius XM, in what appears to be the company’s latest push into making its vehicles’ entertainment systems more feature-rich. The new Sirius XM offer will likely be appreciated by owners of the company’s vehicles, especially considering that the service is among the most popular satellite radios in the country today.
Tesla announced its new offer in an email sent on Monday. An image that accompanied the communication also teased Tesla’s updated and optimized Sirius XM UI for its vehicles. Following is the email’s text.
“Beginning now, enjoy a free, All Access three-month trial subscription to Sirius XM, plus a completely new look and improved functionality. Our latest over-the-air software update includes significant improvements to overall Sirius XM navigation, organization, and search features, including access to more than 150 satellite channels.
“To access simply tap the Sirius XM app from the ‘Music’ section of your in-car center touchscreen—or enjoy your subscription online, on your phone, or at home on connected devices. If you can’t hear SiriusXM channels in your car, select the Sirius XM ‘Subscription’ tab for instruction on how to refresh your audio.”
Tesla has actually been working on Sirius XM improvements for some time now. Back in June, for example, Tesla rolled out its 2020.24.6.4 update, and it included some optimizations to its Model S and Model X’s Sirius XM interface. As noted by noted Tesla owner and hacker @greentheonly, the source code of this update revealed that the Sirius XM optimizations were also intended to be released to other areas such as Canada.
Interestingly enough, Sirius XM is a popular feature that has been exclusive to the Model S and X. Tesla’s most popular vehicle to date, the Model 3, is yet to receive the feature. One could only hope that Sirius XM integration to the Model 3 may eventually be included in the future. Such an update would most definitely be appreciated by the EV community, especially since some Model 3 owners have resorted to using their smartphones or third-party solutions to gain access to the satellite radio service.
The fact that Tesla seems to be pushing Sirius XM rather assertively to its customers seems to suggest that the company may be poised to roll out more entertainment-based apps in the coming months. Apps such as Sirius XM, Spotify, Netflix, and YouTube, may seem quite minor when compared to key functions like Autopilot, after all, but they do help round out the ownership experience of Tesla owners. In a way, Sirius XM does make sense for Tesla’s next-generation of vehicles, especially the Cybertruck and the Semi, both of which would likely be driven in areas that lack LTE connectivity.
Armored Tesla Cybertruck “War Machine” debuts at Defense Expo 2025
Tesla Megapacks chosen for 548 MWh energy storage project in Japan
