Azdome GS63H Dashcam and Mapillary

All comments, suggestions etc. are welcome. Please contact me at wf-hp@gmx.net.

I have an Azdome GS63H dashcam (very high resolution of 3840x2160, they call it "4k") and want to contribute to the crowdsourced street view project called mapillary http://mapillary.org
There has been a legacy web uploader (not working any more) and there is a desktop uploader that works only with movies from one type of dashcam. So I had to find a way to read the GPS data that is included in the .MP4 file and as a result get images with the coordinates in them that can be used by mapillary.
I found the mapillary tools. They are written in Python and work perfectly well in Linux (latest Debian stable running here). However, they

I can warmly recommend the Azdome GS63H.

• Full "4k" resolution at 24 fps
  

• Wifi

• good picture quality in day and night

• costs around 59 USD

How to get your dashcam images into Mapillary?

Preparation

install nvtk_mp42gpx link

install mapillary tools link with documentation

Install python stuff and mapillary tools (as root), everything after that can (and should!) be done as a normal non-root user.

# apt-get install python-pip
# pip install git+https://github.com/mapillary/Piexif
# pip install --upgrade git+https://github.com/mapillary/mapillary_tools


To see the help on the command line the scheme is for all commands the same (example with the upload command)
See the normal help for each command
mapillary_tools upload -h
See the advanced help for each command
mapillary_tools upload -h --advanced
The order of the parameters makes no difference.

Let the fun begin with mapillary tools

Create a directory in your $HOME
$ mkdir -p $HOME/mapillary/video-files/
$ mkdir -p $HOME/mapillary/img/

Copy your files to your harddisk
$ cp /mnt/your_sdcard_directory/*.MP4 $HOME/mapillary/video-files

change into your dashcam video directory and extract the GPS information into gpx files (the mapillary tools cannot read GPS data from the MP4-files directly).
Rename the .gpx files to remove the MP4 extension from them (necessary for some windows dashcam movie players to recognise the corresponding .gpx file automatically).
$ cd $HOME/mapillary/video-files
$ for i in *.MP4; do nvtk_mp42gpx.py -i $i -o $i.gpx -f; done; mmv '*.MP4.gpx' '#1.gpx'

I describe all the single steps with mapillary-tools, you can combine them with the commands listed below

Step 1 - Images from Video

Extract the images from the mp4-file
mapillary_tools sample_video --advanced --import_path "$HOME/mapillary/img/" --video_sample_interval .5 --video_import_path "/mnt/videos/dashcam/mapillary-upload/selection/"

use --video_sample_interval .1 if you want lots of data (".1" will give you 1811 images for 3 minutes of video).

Step 2 - geotag images from .gpx

Extract the GPS information from the gpx files we created before.
My GPS coordinates are ca. 600 ms later than the movie.
mapillary_tools process --advanced --user_name yourusername --import_path "$HOME/mapillary/img/" --offset_time "-.6" --duplicate_distance 0.5 --geotag_source gpx --geotag_source_path "/mnt/videos/dashcam/mapillary-upload/selection/2019_0610_170345_164.gpx"

the same in a loop for all directories
for i in `ls -1 $HOME/mapillary/img/mapillary_sampled_video_frames/`; do mapillary_tools process --advanced --user_name yourusername --import_path "$HOME/mapillary/img/mapillary_sampled_video_frames/$i" --offset_time "-.6" --duplicate_distance 0.5 --geotag_source gpx --geotag_source_path "/mnt/videos/dashcam/mapillary-upload/selection/$i.gpx"; done
When first using the parameter --user_name the program asks you for your account data. This is - if I understood this correctly - embedded in an EXIF tag (of course not in cleartext), so the mapillary system recognises after uploading from which user the images were uploaded.

Step 3 - Upload

Finally bulk upload the images in import_path and its subdirectories.
mapillary_tools upload --import_path "$HOME/mapillary/img/" --move_uploaded
or one of the subdirectories

Uploading might take some time, depending on your bandwith (mine is definitely too low ;) ). First the command scans all the images and selects only those with valid EXIF tags. This step might take some minutes (or hours, depending how many pictures you upload).
At least I could reduce the data usage, from around 50GB in videos to around 25GB in images, and I didn't have to manually start the upload for each video file in the legacy uploader, just had to let the linux box run for some days.
Another advantage is the automatic retry in case of connection problems for individual images.
And finally, if the upload is interrupted, the mapillary tools remember which images have already been uploaded and continue the upload where it was interrupted.

Combination of commands

The single commands can be combined:
sample_video + process = video_process
process + upload = process_and_upload
sample_video + process + upload = video_process_and_upload

Troubleshooting

Problem:

If you get an error like
AccessDenied: Invalid according to Policy: Policy expired
(especially after updating python, pip, or mapillary tools) it is possible that your config file / password / hash / whatever that was written in your jpg-files doesn't fit the current version of the server.

Solution:

Delete the config file in ~/.config/mapillary/configs/
enter again your email and password
do a process --rerun
then uploading works again

mapillary-tools in Windows

Setting up everything

If you are not so lucky to have a Debian box available, how can you run the mapillary-tools in Windows? - yes, of course

Install python:

Install nvtk_mp42gpx

Install mapillary tools

Attention:
I did not manage to get mapillary-tools to accept the username/password in windows. I started patching the source code, but probably some library is missing, so I took the easy path and just set username/password in Linux and copied the config file the windows computer.
You can find it in Linux at
~/.config/mapillary/configs/MkJKbDA0bnZuZlcxeTJHTmFqN3g1dzo1YTM0NjRkM2EyZGU5MzBh.
It should be copied to
C:\Users\\.config\mapillary\configs in Windows.
The filename can be different in windows, just have a look how the file was named by mapillary-tools and use the same filename.

Running everything

1)
cd C:/mapillary/video-files
for i in *.MP4; do nvtk_mp42gpx.py -i $i -o $i.gpx -f; done
for i in *.MP4.gpx; do mv "$i" "${i/.MP4.gpx}".gpx; done


2)
mapillary_tools sample_video --advanced --import_path "C:/mapillary/img/" --video_sample_interval .5 --video_import_path "C:\mapillary\video-files"

3)
for j in `ls -1 /c/mapillary/img/mapillary_sampled_video_frames/`; do echo $i; sleep 1; i=$(sed -e 's/\///' <<< $j); mapillary_tools process --advanced --user_name myusername --import_path "/c/mapillary/img/mapillary_sampled_video_frames/$j" --offset_time "-.6" --duplicate_distance 0.1 --geotag_source gpx --geotag_source_path "/c/mapillary/video-files/$i.gpx" --rerun; done

Check the images now, if you really want to upload them!!!

4)
mapillary_tools upload --import_path /c/mapillary/img/mapillary_sampled_video_frames/ --move_uploaded --number_threads 1

Written 2019-07-29 - last update 2020-06-23