Skip to main content
Version: Next

Visualization Tool

This tutorial is about DriveLangVis (DLVis for short), the visualization tool for Bench2Drive-VL dataset (or other dataset generated by DriveCommenter / inference results from B2DVL / evaluation results from B2DVL).

Since B2DVL dataset is complicated - images, annotations and VQAs are saved under different folders, we developed a tool to show them frame by frame.

screenshot

As shown in this screenshot, DLVis shows two images on the top. In VQA View mode, DLVis lists VQAs at below. In Full View mode, DLVis shows annotation data from Bench2Drive dataset originally, appended annotation data from DriveCommenter, then come the VQAs.

DLVis can both be used to view the dataset and to edit it.

removed feature

In order to support multiple users to work together, DLVis used to support a user system. However that was somehow buggy and is removed now.

Starting up​

To start DLVis, you should write a config file first. Create a config.yaml under DriveLangVis:

config.yaml
# config.yaml
appendix_path: "path/to/project/output/vqagen/gt+front_cam/appendix"
# appended data generated by DriveCommenter
qa_path: "path/to/project/output/vqagen/gt+front_cam"
# vqas generated by DriveCommenter

anno_image_path: "path/to/project/eval_v1/gt+front_cam"
anno_rel_path: "camera/anno_rgb_front"
anno_subfix: "jpg"
# anno_image_path/{scenario_name}/anno_rel_path/{frame:05d}.anno_subfix shows on top-left

b2d_data_path: "path/to/project/eval_v1/gt+front_cam"
b2d_rel_path: "camera/anno_bev"
b2d_subfix: "jpg"
# b2d_image_path/{scenario_name}/b2d_rel_path/{frame:05d}.b2d_subfix shows on top-right
# DLVis will also search for original annotation data (the same format as Bench2Drive dataset) under b2d_data_path.

After that, start DLVis:

cd ./DriveLangVis
python edit_webui.py

Now open 127.0.0.1:5000 in your browser, you can find DLVis running.

If it is the first time you open DLVis, Relative Path field should be empty, no images and data would be shown below. Now click on Check out overview and you will see this in a new tab:

overview

These are all scenarios found in the path you given in config file. Now copy one of them, for example, RouteScenario_2050_rep0_Town12_SignalizedJunctionRightTurn_Weather8_06_07_01_50_58, and paste it in Relative Path field, then click on load.

Now the scenario will be loaded, starting from frame 0.

loaded scenario

Feel free to play around!

Edit the dataset​

If you want to edit the dataset, the best practice might be:

  1. Load the scenario you want to edit (please refer to previous section).

  2. Now you're at the first frame of targeted scenario. Now you should start to check all contents you want to edit. If you only want to edit VQAs, it's better to switch to VQA view.

switch views

vqa view This means you're in vqa view. full view This means you're in full view.

  1. Find the wrong content, and modify it by clicking on the Edit button besides it. Modify it in the input field, and click on Submit. Now it is edited as you wish!
Extra Features

DLVis provides several features to facilitate your dataset editing process.

  • Sometimes you might not be able to make a clear decision, you can click on Mark as Controversial to leave it for later judgements.

  • For some questions, there might exists a common answer. You can save that answer by clicking Add to Common Options. By doing so, when you're dealing with the same question after that, the option you added will appear below, and you can directly select it to update the answer.

  • You can click on View History to check the edit history of a value.

edit screenshot

  1. When you finishes editing a whole frame, you can click on Marked as Verified to mark your progress.
controversy

If any value in this frame is marked as controversial, this frame is marked as controversy.

  1. Check the next frame...

SSH forwarding​

You probably faces a situation that your data is on a remote machine. A best practice it to serve DLVis on the remote machine, and forward the remote port to your local machine. For example, if your DLVis runs on the default 5000 port:

ssh -L 8080:localhost:5000 username@rempote_ip

Now open 127.0.0.1:8080 on your local machine, you can visit your remote dataset.

Function details​

Entry Frame & Exit Frame​

Marks the starting and ending frames of the data to be retained. If you need to discard data with significant errors at the beginning or end, make the adjustments here. Only the frame interval [entry frame, exit frame) will be preserved for further processing.

Overview​

Located at the top of the page, below WebUI Debug. Click Check out overview to view the overall annotation status of the dataset, including the labeling status of each frame in every scenario subset. This helps you keep track of the overall progress.

Prev & Next Buttons (Frequently Used)​

Located on the left side of the top bar, these buttons allow you to directly switch to the previous or next frame.

Frame History​

Located on the top bar. Click View History to see all modification records of the current frame.

View Switch​

Located on the top bar. Click Switch to VQA/Full View to switch between views.

  • VQA View shows only the VQA content.
  • Full View shows both the original sensor data collected by B2D and the flags generated by DriveCommenter.

Status Annotation (Frequently Used)​

Each frame can be marked as:

  • "raw" (unprocessed),
  • "controversial", or
  • "verified".

If any item in a frame is marked as "controversial", the entire frame will be marked as "controversial" and cannot be changed unless the controversial flag is removed.

Filter​

Each of the three JSON sections has a filter input box beneath it. You can enter keywords to adjust which items are displayed, following these rules:

  1. word: Shows items with a key exactly matching word. If a parent item matches, all its children are also displayed.

  2. word*: Shows items whose key starts with word. If the parent key starts with word, all children are retained.

  3. *word: Shows items whose key ends with word. If the parent key ends with word, all children are retained.

  4. *word*: Shows items whose key contains word as a substring. If the parent key contains word, all children are retained.

  5. A B: A and B follow any of the above four rules. Displays items that match either rule. If a parent matches, all children are shown.

  6. C D: C and D follow any of the above five rules. Displays items that match either rule. If a parent matches, all children are shown.

  7. Leave blank to disable filtering and show all items.

Example: *id *affect* steer Shows items whose keys end with id, contain the substring affect, or exactly match steer. If a parent item matches, all children are shown.

QID Filter (Frequently Used)​

Located on the left side of the bottom bar. Enter the qid of the VQA question you want to view. Leave blank to show all.

Example: 18 24 25 Only questions with qid 18, 24, and 25 will be displayed.

Hide/Show Key Object Info​

Located on the left side of the bottom bar. Toggles the collapse/expand state of key_object_info.

Stick Images​

Located on the bottom bar. Keeps the image persistently pinned at the top of the page.