Creating Fingerprints

From comma wiki
Jump to: navigation, search

OpenPilot recognizes which car is connected by comparing the CAN signals read to lists in... /data/openpilot/common/fingerprints.py

Currently, different trim packages of the same model often have different fingerprints, which can require one to create their own fingerprint if Comma hasn't already included it.

Although there are JavaScripts for pulling fingerprints from .csv logs (downloaded from a Cabana drives) via Node.js, they usually don't work especially because Cabana drives normally begin well after the car is started, and fingerprints are more about the DBC messages seen during vehicle start-up.

I generally recommend having at least 2 sessions open, SSH'd into the NEOS device. This procedure does NOT teach you how to install OpenPilot, SSH, use VIM, etc.


I. Turn off car, connect Panda to car (normally via Giraffe), and connect Panda to NEOS device (eg. EON or NEO) running OpenPilot.


II. Obtain DBC messages for fingerprint using one of 2 methods (A or B).

Method A. Python script (based on data @rickbias provided... thanks!)

1. Create a file... /data/openpilot/selfdrive/get_fingerprint.py

#!/usr/bin/env python

import zmq
import selfdrive.messaging as messaging
from selfdrive.services import service_list

context = zmq.Context()
logcan = messaging.sub_sock(context, service_list['can'].port)
msgs = {}
while True:
  lc = messaging.recv_sock(logcan, True)
  for c in lc.can:
    if c.src == 0:
      msgs[c.address] = len(c.dat)
  out = []
  for key in sorted(msgs):
    out.append(str(key) + "L: " + str(msgs[key]))
  print ""
  print "number of messages:", len(out)
  print "fingerprint", ", ".join(out)

Don't forget to make it executable...

chmod +x /data/openpilot/selfdrive/get_fingerprint.py

2. Run these commands in 2 separate sessions (SSH'd into NEOS)...

/data/openpilot/selfdrive/boardd/boardd
cd /data/openpilot/selfdrive && PYTHONPATH=/data/openpilot PREPAREONLY=1 /data/openpilot/selfdrive/get_fingerprint.py

3. Turn on the car's ignition, and wait up to ~20 seconds to ensure all the appropriate DBC messages are seen, like this...

number of messages: 107
fingerprint 36L: 8, 37L: 8, 166L: 8, 170L: 8, 180L: 8, 295L: 8, 296L: 8, 426L: 6, 452L: 8, 466L: 8, 467L: 8, 550L: 8, 552L: 4, 560L: 7, 562L: 6, 581L: 5, 608L: 8, 610L: 8, 614L: 8, 643L: 7, 658L: 8, 713L: 8, 740L: 5, 742L: 8, 743L: 8, 800L: 8, 810L: 2, 814L: 8, 824L: 2, 829L: 2, 830L: 7, 835L: 8, 836L: 8, 863L: 8, 869L: 7, 870L: 7, 871L: 2, 898L: 8, 900L: 6, 902L: 6, 905L: 8, 913L: 8, 918L: 8, 921L: 8, 933L: 8, 944L: 8, 945L: 8, 950L: 8, 951L: 8, 953L: 8, 955L: 8, 956L: 8, 971L: 7, 974L: 8, 975L: 5, 993L: 8, 998L: 5, 999L: 7, 1000L: 8, 1001L: 8, 1014L: 8, 1017L: 8, 1020L: 8, 1041L: 8, 1042L: 8, 1044L: 8, 1056L: 8, 1057L: 8, 1059L: 1, 1071L: 8, 1076L: 8, 1077L: 8, 1082L: 8, 1083L: 8, 1084L: 8, 1085L: 8, 1086L: 8, 1114L: 8, 1132L: 8, 1161L: 8, 1162L: 8, 1163L: 8, 1164L: 8, 1165L: 8, 1166L: 8, 1167L: 8, 1175L: 8, 1227L: 8, 1228L: 8, 1235L: 8, 1237L: 8, 1279L: 8, 1552L: 8, 1553L: 8, 1556L: 8, 1557L: 8, 1568L: 8, 1570L: 8, 1571L: 8, 1572L: 8, 1595L: 8, 1777L: 8, 1779L: 8, 1904L: 8, 1912L: 8, 1990L: 8, 1998L: 8

4. <CTRL> + 'C' to break out of processes still running in each session.


Method B. Obtain DBC messages semi-manually

1. Comment out (insert # at beginning of line) controlsd in 'managed_processes' section of... /data/openpilot/selfdrive/manager.py

2. Run this in NEOS...

cd /data/openpilot/selfdrive && PYTHONPATH=/data/openpilot PREPAREONLY=1 /data/openpilot/selfdrive/controls/controlsd.py

You'll see something like this...

make: Nothing to be done for 'all'.
make: Nothing to be done for 'all'.
make: Nothing to be done for 'all'.
pid 9611's current scheduling policy: SCHED_OTHER
pid 9611's current scheduling priority: 0
pid 9611's new scheduling policy: SCHED_FIFO
pid 9611's new scheduling priority: 3
waiting for fingerprint...

3. Turn on the car's ignition. After Panda initializes, numerous signals will appear. Assuming the car isn't recognized by OpenPilot yet, output like this can be expected...

car doesn't match any fingerprints: {898L: 8, 900L: 6, 902L: 6, 905L: 8, 1162L: 8, 810L: 2, 1164L: 8, 1165L: 8, 1166L: 8, 1167L: 8, 1552L: 8, 1553L: 8, 1042L: 8, 1556L: 8, 1557L: 8, 918L: 8, 1175L: 8, 998L: 5, 1161L: 8, 1568L: 8, 944L: 8, 1570L: 8, 1059L: 1, 36L: 8, 37L: 8, 166L: 8, 295L: 8, 552L: 4, 170L: 8, 814L: 8, 560L: 7, 945L: 8, 562L: 6, 180L: 8, 1077L: 8, 950L: 8, 951L: 8, 824L: 2, 1076L: 8, 1082L: 8, 1595L: 8, 956L: 8, 829L: 2, 296L: 8, 800L: 8, 863L: 8, 1163L: 8, 452L: 8, 581L: 5, 1990L: 8, 1057L: 8, 713L: 8, 955L: 8, 1228L: 8, 467L: 8, 466L: 8, 1235L: 8, 1237L: 8, 1114L: 8, 933L: 8, 608L: 8, 993L: 8, 610L: 8, 1083L: 8, 740L: 5, 550L: 8, 614L: 8, 871L: 2, 1000L: 8, 1001L: 8, 999L: 7, 1132L: 8, 658L: 8, 1777L: 8, 1779L: 8, 1014L: 8, 1044L: 8, 1020L: 8, 426L: 6, 1279L: 8}
Traceback (most recent call last):
  File "/data/openpilot/selfdrive/controls/controlsd.py", line 542, in <module>
    main()
  File "/data/openpilot/selfdrive/controls/controlsd.py", line 539, in main
    controlsd_thread(gctx, 100)
  File "/data/openpilot/selfdrive/controls/controlsd.py", line 455, in controlsd_thread
    raise Exception("unsupported car")
Exception: unsupported car

4. Copy the text between the curly braces {} into a new file 'dbc_messages.csv' (if you already created this, then put the messages into a new line), bearing in mind this is likely an incomplete fingerprint which will require additional work later.

5. Run this in NEOS (temporarily converts commas into newlines, sort unique values, etc.) to get a proposed fingerprint...

tr , '\n' < dbc_messages.csv|sort -ug|tr '\n' ,


III. Document the fingerprint

1. Copy the DBC messages obtained from section II into the "_FINGERPRINTS" section of... /data/openpilot/common/fingerprints.py [Create new sub-section for car, or overwrite pre-existing fingerprint of similar car.]

2. Remove the pre-existing, compiled Python file in NEOS...

cd data/openpilot/common && rm fingerprints.pyc

3. Re-compile a new one in NEOS...

python -m compileall .

4. If you used section II Method A, you're almost done... so go to section IV. If you used Method B, keep reading... You should be able to keep your car on while looping through section II Method B & section III (steps 1-3) until controlsd.py says your fingerprint matches. With every loop, you'll want to overwrite the previous fingerprint attempt in fingerprints.py w/ the new relevant output from section II Method B step 5. You might need to loop through these steps dozens of times before success like this...

fingerprinted TOYOTA PRIUS 2017
ECU Camera Simulated:  False
ECU DSU Simulated:  False
ECU APGS Simulated:  False
{'event': 'alert_add', 'alert_type': 'startup', 'enabled': False}
set CarParams
MainProcess lagging by 8.71 ms
MainProcess lagging by 2.26 ms
MainProcess lagging by 2.08 ms
...

5. Don't forget to revert the change from section II Method B step 1...

vim /data/openpilot/selfdrive/manager.py

[Remove the # inserted earlier that commented out controlsd in the 'managed_processes' section.]


IV. Turn off car's ignition, then cycle NEOS...

reboot