Tesla CCS candidate check - Round 7
Denver Gingerich
denver at sfconservancy.org
Fri May 18 17:54:58 UTC 2018
These are the results of the CCS check we did on the Tesla candidate
source code made available at https://github.com/teslamotors/buildroot
in May 2018. This is the seventh source candidate we have received from
Tesla.
In this check, per the GPL's requirements, we are seeking to both (a)
confirm that we are able to build binaries from the source that
correspond to the binaries Tesla ships and (b) verify that we are able
to replace the binaries Tesla ships with our own (possibly modified)
binaries, i.e. using installation instructions that Tesla might provide.
Below are the binaries Tesla ships in its Model S that we are comparing
to built binaries. These comparison binaries were provided to us by a
Model S car owner who was one of the many to report this violation to us:
* firmware_2.12.126 - a filesystem provided by Parrot
* 2.15.50.img.bz2 - a filesystem provided by Nvidia (presumed to match
the now-named Tesla Autopilot 2 platform - we'll refer to this as the
"main board" system in this report)
First, here are the details on how we downloaded and prepped the source
candidate Tesla provided to us:
We obtained the source candidate using the following steps (the checkout
step is necessary to ensure the reader is using the same version of the
code that we did - it was the latest commit at the time of this writing):
$ git clone https://github.com/teslamotors/buildroot.git
$ cd buildroot
$ git checkout fb9fa1f65c0ffb50695d620d74249b3ca10f7583
Note that the above buildroot repository automatically retrieves via
reference the repository at https://github.com/teslamotors/linux , which
is analyzed below in the "linux (part 1)" section.
== Sources for one of the Linux installs found in vehicle binaries
appear to be completely missing ==
It appears that this buildroot repository contains only code intended to
match the main board binaries. Email to users seen at
https://teslamotorsclub.com/tmc/threads/tesla-releases-some-gpl-opensource-bits.115545/
specifically mentioned Nvidia in relation to this release, and that
"Work is underway on preparing sources in other areas as well". While
it does not appear that this repository contains any code that
corresponds to the Parrot binaries, we expect Tesla will provide it
later. Tesla has previously privately provided to us two separate build
trees in its past source candidates, one for "Nvidia" and one for
"Parrot" (each containing its own copy of Linux and associated userspace
tools).
When provided, we expect these sources to contain a separately
configured, different copy of Linux, and include the source code for the
88w8688_uap.ko and 88w8688_uap_mlan.ko files that appear in the Tesla
Model S.
== Check summary of sources provided ==
We were able to generate some binary files after making several changes
to the build scripts. We have already publicly provided pull requests,
since the availablity of these repositories on GitHub indicate that is
an appropriate method for receiving improvements for compliance of these
sources. These pull requests are at
https://github.com/teslamotors/buildroot/pulls , and we hope to provide
more as this work continues.
However, even with these changes, the candidate does not appear to be
complete because it is both (a) missing installation instructions and
(b) missing a few details on how to build the initramfs.
It appears that the latter could be resolved with a few extra notes in
the README.Tesla file (see below), especially pertaining to the
ap-hw2i_defconfig file (see the "initramfs" section below). This could
even be provided by someone more familiar with buildroot, and may not
require a Tesla employee.
However, the former (installation instructions) can likely only be
provided by Tesla, since the methods for installing software onto Tesla
hardware is likely highly specific to that particular hardware.
Also, we would recommend that Tesla host all the files required by its
buildroot repository rather than having buildroot download them from
third-party locations, as those sources may become unavailable in the
future, which would cause further, future compliance problems for Tesla.
The main board binaries we checked against do not match what was built
by the source repository (see the last section below). So there need to
be corrections and/or additions made to future candidates to ensure the
source does indeed correspond to the binaries.
== Initial build process ==
We found a README file in the root directory of the repository, as well
as a README.Tesla file. It was clear that the README file contained
generic buildroot instructions so we then read the README.Tesla file,
which stated:
"Current branch is buildroot-ap, it contains default configs to build
packages
for the Tesla Autopilot 2 platforms:
ap-hw2_defconfig
ap-hw25_defconfig"
Since we found we were on the builtroot-ap branch already (prior to the
"git checkout", "git branch -a" showed the current branch to be
builtroot-ap), we then looked for the above _defconfig files. We found
them in the "configs" directory, and then copied
configs/ap-hw25_defconfig to .config so that we could use that
configuration for the build process. (Note that the ap-hw2_defconfig
and ap-hw25_defconfig configuration files differ only in which tag they
checkout from the kernel tree.)
Note that the README files did not specify any particular operating
system that one should use to build the project, so we used the
operating system we had on-hand, which was an amd64 version of Debian 9.
When we then ran "make", we were prompted with an ncurses-rendered
screen entitled "Buildroot 2016.05-00004-gfb9fa1f65 Configuration" with
some configuration options. Since we wanted to use the defaults from
ap-hw25_defconfig, we chose the "Exit" option and hit Enter (answering
Yes to "Do you wish to save your new configuration?").
Since that "make" invocation had completed, we ran "make" again in order
to start building the project using the configuration we just saved.
This did build a few packages without issue, but we then ran into a
number of packages that required fixes in order to build correctly:
== Specific build/package issues ==
=== host-gcc-final-5.3.0 ===
The first time we attempted the build, we received this error:
build: In file included from ../../gcc/cp/except.c:1023:0:
build: cfns.gperf: In function 'const char* libc_name_p(const char*,
unsigned int)':
build: cfns.gperf:101:1: error: 'const char* libc_name_p(const char*,
unsigned int)' redeclared inline with 'gnu_inline' attribute
build: cfns.gperf:26:14: note: 'const char* libc_name_p(const char*,
unsigned int)' previously declared here
A bit of research showed that this error could be solved by the
following patch:
https://gcc.gnu.org/git/?p=gcc.git;a=commitdiff;h=ec1cc0263f156f70693a62cf17b254a0029f4852
We first removed the ChangeLog portion of the patch since it was not
needed to correct the functional issue, then applied the patch, which
applied cleanly (inside both the output/build/host-gcc-final-5.3.0 and
output/build/host-gcc-initial-5.3.0 directories).
=== metis-5.1.0 ===
The build continued after the above patches, but was once again halted:
1: [100%] Built target cmpfillin
1: make[3]: Leaving directory
'.../buildroot/output/build/metis-5.1.0/build/Linux-x86_64'
1: .../buildroot/output/host/usr/bin/cmake -E cmake_progress_start
.../buildroot/output/build/metis-5.1.0/build/Linux-x86_64/CMakeFiles 0
1: make[2]: *** No rule to make target 'w'. Stop.
1: make[2]: Leaving directory
'.../buildroot/output/build/metis-5.1.0/build/Linux-x86_64'
We found after some research that the following patch solved it:
--- a/output/build/metis-5.1.0/Makefile 2013-03-30 12:24:45.000000000 -0400
+++ b/output/build/metis-5.1.0/Makefile 2018-05-17 22:02:00.965427439 -0400
@@ -64,7 +64,7 @@
@if [ ! -f $(BUILDDIR)/Makefile ]; then \
more BUILD.txt; \
else \
- make -C $(BUILDDIR) $@ $(MAKEFLAGS); \
+ make -C $(BUILDDIR) $@ MAKEFLAGS=$(MAKEFLAGS); \
fi
uninstall:
=== initramfs ===
With the above patch, the build continued, but was then halted again:
>>> linux-initramfs 1 Syncing from source dir
.../buildroot/package/linux-initramfs
>>> linux-initramfs 1 Configuring
>>> linux-initramfs 1 Building
cp .../buildroot/output/build/linux-initramfs-1/../../Makefile
.../buildroot/output/build/linux-initramfs-1/output
cp: cannot stat
'.../buildroot/output/build/linux-initramfs-1/../../Makefile': No such
file or directory
package/pkg-generic.mk:195: recipe for target
'.../buildroot/output/build/linux-initramfs-1/.stamp_built' failed
This was somewhat expected, as README.Tesla included this text:
"There's also a separate defconfig used to build an initramfs used with
the kernel
that the above configs use:
ap-hw2i_defconfig"
However, when we tried to build the buildroot project using
ap-hw2i_defconfig as the .config, we received the following error:
Makefile:472: *** util-linux is in the dependency chain of lvm2 that
has added it to its _DEPENDENCIES variable without selecting it or
depending on it from Config.in. Stop.
We were unable to quickly determine a resolution to this problem, so we
omitted the initramfs build by running "touch
output/build/linux-initramfs-1/.stamp_built".
=== protobuf3 ===
After removing the initramfs package from the build per above, we ran
into this error next time we ran make:
buildroot-dl-wrapper: 2018-05-18 07:36:57 (7.55 MB/s) -
'.../buildroot/output/build/.protobuf3-v3.1.0.tar.gz.L3ywCw/output'
saved [4051540]
buildroot-dl-wrapper: ERROR: protobuf3-v3.1.0.tar.gz has wrong sha256 hash:
buildroot-dl-wrapper: ERROR: expected:
0a0ae63cbffc274efb573bdde9a253e3f32e458c41261df51c5dbc5ad541e8f7
buildroot-dl-wrapper: ERROR: got :
fb2a314f4be897491bb2446697be693d489af645cb0e165a85e7e64e07eb134d
buildroot-dl-wrapper: ERROR: Incomplete download, or man-in-the-middle
(MITM) attack
buildroot-dl-wrapper: exit: 1 [1s elapsed, 0 pending]
It appeared from our subsequent research that this issue was benign, so
we edited the hash in package/protobuf3/protobuf3.hash to match the
"got" hash above and then re-ran "make".
=== linux (part 1) ===
This time the build failed with the following:
buildroot-dl-wrapper: PWD: .../buildroot
buildroot-dl-wrapper: Starting: ['support/download/dl-wrapper', '-b',
'git', '-o',
'.../buildroot/output/dl/linux-internal-055213a7808f31ee04299c2d119b15a080b631f6.tar.gz',
'--', 'git at github.com:teslamotors/linux.git',
'internal-055213a7808f31ee04299c2d119b15a080b631f6',
'linux-internal-055213a7808f31ee04299c2d119b15a080b631f6']
(buildroot-dl-wrapper)
buildroot-dl-wrapper: Doing shallow clone
buildroot-dl-wrapper: Cloning into
'linux-internal-055213a7808f31ee04299c2d119b15a080b631f6'...
buildroot-dl-wrapper: fatal: Could not read from remote repository.
buildroot-dl-wrapper: Please make sure you have the correct access rights
buildroot-dl-wrapper: and the repository exists.
It seemed fairly clear from the above that the path to the GitHub URL
was not correct for someone downloading it without a GitHub account (due
to the "access rights" comment) so we changed the .config file as follows:
-BR2_LINUX_KERNEL_CUSTOM_REPO_URL="git at github.com:teslamotors/linux.git"
+BR2_LINUX_KERNEL_CUSTOM_REPO_URL="https://github.com/teslamotors/linux.git"
=== linux (part 2) ===
We then ran into this:
linux-internal-055213a7808f31ee04299c2d119b15a080b631f6+Image:
./scripts/gen_initramfs_list.sh: Cannot open
'.../buildroot/output/images/initramfs.cpio.xz'
linux-internal-055213a7808f31ee04299c2d119b15a080b631f6+Image:
usr/Makefile:73: recipe for target 'usr/initramfs_data.cpio.gz' failed
linux-internal-055213a7808f31ee04299c2d119b15a080b631f6+Image: make[2]:
*** [usr/initramfs_data.cpio.gz] Error 1
linux-internal-055213a7808f31ee04299c2d119b15a080b631f6+Image:
Makefile:963: recipe for target 'usr' failed
Per the initramfs issues above, this is somewhat expected, so we just
ran "touch output/images/initramfs.cpio.xz" and continued (see above
"initramfs" section for details).
== Final result and missing installation instructions ==
After the above change was made, the build finished, but we weren't able
to find any installation instructions. We did find a few files in the
output/images directory, specifically:
* Image
* rootfs.squashfs
The above files appeared to contain the results of the Linux build (the
strings of the Image file included "Linux version 3.18.69-rt75 (...)
(gcc version 5.3.0 (Buildroot 2016.05-00004-gfb9fa1f65-dirty) ) #1 SMP
PREEMPT RT ...") and the rest of the build (the rootfs.squashfs
contained BusyBox and other binaries).
While it is possible we could install the above files onto Tesla
hardware with some guessing, it would be very difficult to do, and since
the GPL requires installation instructions anyway, this omission does
still require resolution by someone at Tesla with adequate expertise.
== Comparison with Model S binaries ==
We compared the binaries built above with those we found in the binaries
shipped on the Tesla Model S and discovered that they did not match. In
particular, we found the following issues:
* BusyBox v1.24.2 was provided via the buildroot project
* BusyBox v1.19.2 (built 2016-02-23) was shipped on the Model S
So we don't appear to have a source candidate that correponds to files
shipped on the Model S.
We chose not to compare other binaries inside the 2.15.50.img.bz2 binary
file (for the main board system) since the BusyBox version was already
so different. In order for the source candidate to be compliant, the
binaries that it builds (including but not limited to BusyBox binaries)
must correspond to the binaries that Tesla ships.
We do note that Tesla has indicated these sources are for their 2018.12
build. We are aware that we may be comparing binaries from an earlier
build, and that explains the mismatches. We ask Tesla to indicate more
comprehensively their plans for compliance on past distributions of
older versions of the software. We are willing to help figure out what
compliance paths are feasible, and on our part may be willing to forgive
past violations for older versions that may never resolved, provided
that Tesla can attest that distribution of those older binaries has
definitely ceased.
(For example, we're particularly concerned about what binaries may be in
the physical cars that have not received online updates yet. Compliance
for those binaries, which as far as we can tell would match the binaries
we have in our possesion, is an important issue.)
Denver Gingerich
Software Freedom Conservancy
More information about the ccs-review
mailing list