Skip to content

Resolve SD Card Image Creation #21

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
CalebProvost opened this issue Apr 12, 2021 · 9 comments
Closed

Resolve SD Card Image Creation #21

CalebProvost opened this issue Apr 12, 2021 · 9 comments
Assignees
Labels
Milestone

Comments

@CalebProvost
Copy link
Contributor

DoD: Fix SD Card Image creation (blocker to #15 (comment))

@CalebProvost CalebProvost added the todo Work which needs to be done label Apr 12, 2021
@CalebProvost CalebProvost added this to the Sprint 3 milestone Apr 12, 2021
@CalebProvost CalebProvost self-assigned this Apr 12, 2021
@CalebProvost
Copy link
Contributor Author

Oddly enough... Fresh build results in an image being created... I think the preexisting tools might have an issue with creating an image if one already exists. Inserted a test which will remove the image first before making it.

Testing now.

@CalebProvost
Copy link
Contributor Author

It doesn't appear to be an actual issue... It'll fail inside the docker container... sometimes...
I think we'll need to not automate this step in order to have this step work...
Or... Automate this step outside of the docker container...

I'll work on extracting this step from the docker script and instead insert it into a script which will trigger each docker execution, then the sd card creation.

@CalebProvost CalebProvost added WIP Current "work in progress" bug Something isn't working and removed todo Work which needs to be done WIP Current "work in progress" labels Apr 14, 2021
@CalebProvost
Copy link
Contributor Author

I still have yet to determine the reasoning, but the dosdcard.sh script we're using to create the tegra's sd card image doesn't want to hand off the GPT information to the tegrra210-flash-helper.sh script which creates our image. However, because it does work outside of docker, I'll write the all-integrated-run script to to do a docker container run to build the tegra image, then do the do sd card image, then back to docker to flash the arduino with it's needed firmware.

@CalebProvost
Copy link
Contributor Author

On second thought... because the tegra build takes so long, I'll have the integrated script do the arduino work via docker 1st, then do the tegra image via docker, then (outside of docker) create the sd card image. That way a user can unplug the arduino and wait for the tegra stuff without having to wait for hours if the arduino has issues.

@CalebProvost CalebProvost added idea/discussion Not work to be done; but might be and removed bug Something isn't working labels Apr 16, 2021
@CalebProvost
Copy link
Contributor Author

Potential new build script for yocto:

#!/bin/bash

# Set Defaults if not provided
CUR_DIR=$(pwd)
# For Yocto Build
[ -z "${MACHINE}" ] && export MACHINE="jetson-nano-2gb-devkit"
[ -z "${BRANCH}" ] && export BRANCH="master"
[ -z "${DISTRO}" ] && export DISTRO="final-project"
[ -z "${BUILD_IMAGE}" ] && export BUILD_IMAGE="demo-image-full"
[ -z "${NVIDIA_DEVNET_MIRROR}" ] && export NVIDIA_DEVNET_MIRROR="file:///home/user/sdk_downloads"

# Clone L4T Yocto Base
YL4T_SUCCESS="false"
git clone https://github.com/OE4T/tegra-demo-distro.git ${CUR_DIR}/tegra-demo-distro
cd "${CUR_DIR}/tegra-demo-distro/" || echo "Could not enter L4T directory"
git checkout ${BRANCH}
git submodule update --init --recursive

# Clone AESD Final Project
git clone https://github.com/cu-ecen-5013/final-project-CalebProvost.git --branch yocto-layer layers/meta-final-project
. ./setup-env --machine ${MACHINE} --distro ${DISTRO}

# Begin OE4T Build
bitbake ${BUILD_IMAGE} && export YL4T_SUCCESS="true"

if [ "${YL4T_SUCCESS}" = "true" ]; then
    mkdir -p "${CUR_DIR}/tegraflash"
    cd "${CUR_DIR}/tegraflash" || echo "Could Not Enter SD Card Staging Directory"
    cp "${CUR_DIR}/tegra-demo-distro/build/tmp/deploy/images/${MACHINE}/${BUILD_IMAGE}-${MACHINE}.tegraflash.tar.gz" "${CUR_DIR}/tegraflash/"
    tar -xf "${CUR_DIR}/tegra-demo-distro/build/tmp/deploy/images/${MACHINE}/${BUILD_IMAGE}-${MACHINE}.tegraflash.tar.gz"
    [ -f "../${BUILD_IMAGE}-${MACHINE}.img" ] && rm -rf "../${BUILD_IMAGE}-${MACHINE}.img"
    ./dosdcard.sh "../${BUILD_IMAGE}-${MACHINE}.img" && SDCARD_IMAGE="true"

    echo "" && echo ""
    echo "####################################################################################################"
    echo "Yocto has finished building the OE4T image \"${BUILD_IMAGE}\" for the \"${MACHINE}\"."
    echo "Deloyment files can be found here: ${CUR_DIR}/tegra-demo-distro/build/tmp/deploy/images/${MACHINE}/"
    if [ "${SDCARD_IMAGE}" = "true" ]; then
        echo ""
        echo "SD Card image for flashing can be found here: ${CUR_DIR}/${BUILD_IMAGE}-${MACHINE}.img"
    else
        echo "Yocto Build of OE4T Complete. SD Card Image creation has failed and is known to do so within Docker."
        echo "Follow the steps echo'ed below to try again (externally from docker)"
        echo 'mkdir -p "$PWD/tegraflash" && cd "$PWD/tegraflash"'
        echo 'cp "$PWD/tegra-demo-distro/build/tmp/deploy/images/${MACHINE}/${BUILD_IMAGE}-${MACHINE}.tegraflash.tar.gz" .'
        echo 'tar -xf "$PWD/tegra-demo-distro/build/tmp/deploy/images/${MACHINE}/${BUILD_IMAGE}-${MACHINE}.tegraflash.tar.gz"'
        echo './dosdcard.sh "../${BUILD_IMAGE}-${MACHINE}.img"'
    fi
    echo "####################################################################################################"
    echo ""
    exit 0

else
    echo "Could not determine the success of Yocto building L4T; exiting..."
    exit 1
fi

@CalebProvost
Copy link
Contributor Author

Caleb Provost 1 hour ago
The output of the working SD Creation:
log.log

aesd@aesdvm:/home/user/build/sdcardprep$ bash -x ./dosdcard.sh ../NAME.img + MACHINE=jetson-nano-2gb-devkit

  • BOARDID=3448

  • FAB=400

  • ./tegra210-flash-helper.sh --sdcard -B 1048576 -s 16G -b demo-image-full flash.xml.in tegra210-p3448-0003-p3542-0000.dtb jetson-nano-2gb-devkit.cfg 0xA4000 '' boot.img demo-image-full.ext4 ../NAME.img

Board ID(3448) version(400) SKU() revision() from environment

New
Dan 32 minutes ago
Looks like there's a tegraflash helper script it's calling that needs bash -x? You need to trace wherever it's failing and add some debug information there, either with bash -x or echo statements or whatever.

@CalebProvost
Copy link
Contributor Author

Caleb Provost 1 hour ago
The output of the working SD Creation:
log.log

aesd@aesdvm:/home/user/build/sdcardprep$ bash -x ./dosdcard.sh ../NAME.img + MACHINE=jetson-nano-2gb-devkit

* BOARDID=3448

* FAB=400

* ./tegra210-flash-helper.sh --sdcard -B 1048576 -s 16G -b demo-image-full flash.xml.in tegra210-p3448-0003-p3542-0000.dtb jetson-nano-2gb-devkit.cfg 0xA4000 '' boot.img demo-image-full.ext4 ../NAME.img

Board ID(3448) version(400) SKU() revision() from environment

New
Dan 32 minutes ago
Looks like there's a tegraflash helper script it's calling that needs bash -x? You need to trace wherever it's failing and add some debug information there, either with bash -x or echo statements or whatever.

The results when following his instructions:

user@1fe92c6db610:~/build/sdprep$ export MACHINE=jetson-nano-2gb-devkit && export BOARDID=${BOARDID:-3448} && export FAB=${FAB:-400}
user@1fe92c6db610:~/build/sdprep$ bash -x ./tegra210-flash-helper.sh --sdcard -B 1048576 -s 16G -b final-project-image  flash.xml.in tegra210-p3448-0003-p3542-0000.dtb jetson-nano-2gb-devkit.cfg 0xA4000 "" boot.img final-project-image.ext4 "../test.img"
+ bup_blob=0
+ keyfile=
+ keyfile_args=
+ spi_only=
+ no_flash=0
+ flash_cmd=
+ sdcard=
+ make_sdcard_args=
+ imgfile=
+ dataimg=
+ inst_args=
+ blocksize=4096
+++ basename ./tegra210-flash-helper.sh
++ getopt -n tegra210-flash-helper.sh -l bup,no-flash,sdcard,spi-only,datafile:,usb-instance: -o u:s:b:B:yc: -- --sdcard -B 1048576 -s 16G -b final-project-image flash.xml.in tegra210-p3448-0003-p3542-0000.dtb jetson-nano-2gb-devkit.cfg 0xA4000 '' boot.img final-project-image.ext4 ../test.img
+ ARGS=' --sdcard -B '\''1048576'\'' -s '\''16G'\'' -b '\''final-project-image'\'' -- '\''flash.xml.in'\'' '\''tegra210-p3448-0003-p3542-0000.dtb'\'' '\''jetson-nano-2gb-devkit.cfg'\'' '\''0xA4000'\'' '\'''\'' '\''boot.img'\'' '\''final-project-image.ext4'\'' '\''../test.img'\'''
+ '[' 0 -ne 0 ']'
+ eval set -- ' --sdcard -B '\''1048576'\'' -s '\''16G'\'' -b '\''final-project-image'\'' -- '\''flash.xml.in'\'' '\''tegra210-p3448-0003-p3542-0000.dtb'\'' '\''jetson-nano-2gb-devkit.cfg'\'' '\''0xA4000'\'' '\'''\'' '\''boot.img'\'' '\''final-project-image.ext4'\'' '\''../test.img'\'''
++ set -- --sdcard -B 1048576 -s 16G -b final-project-image -- flash.xml.in tegra210-p3448-0003-p3542-0000.dtb jetson-nano-2gb-devkit.cfg 0xA4000 '' boot.img final-project-image.ext4 ../test.img
+ unset ARGS
+ true
+ case "$1" in
+ sdcard=yes
+ shift
+ true
+ case "$1" in
+ blocksize=1048576
+ shift 2
+ true
+ case "$1" in
+ make_sdcard_args=' -s 16G'
+ shift 2
+ true
+ case "$1" in
+ make_sdcard_args=' -s 16G -b final-project-image'
+ shift 2
+ true
+ case "$1" in
+ shift
+ break
+ flash_in=flash.xml.in
+ dtb_file=tegra210-p3448-0003-p3542-0000.dtb
+ sdramcfg_file=jetson-nano-2gb-devkit.cfg
+ odmdata=0xA4000
+ boardcfg_file=
+ kernfile=boot.img
+ imgfile=final-project-image.ext4
+ shift 7
+++ dirname ./tegra210-flash-helper.sh
++ readlink -f .
+ here=/home/user/build/sdprep
+ flashapp=/home/user/build/sdprep/tegraflash.py
+ '[' -e ./flashvars ']'
+ . ./flashvars
++ mktemp cvm.bin.XXXXX
+ cvm_bin=cvm.bin.a5gxF
+ '[' -z 3448 -a -z 400 ']'
+ boardid=3448
+ boardver=400
+ boardsku=
+ boardrev=
+ echo 'Board ID(3448) version(400) SKU() revision() from environment'
Board ID(3448) version(400) SKU() revision() from environment
+ '[' 3448 = 3448 ']'
+ expr 400 '<' 300
+ dtbfab=b00
+ '[' -z '' ']'
+ boardsku=0000
+ BOARDSKU=0000
+ '[' -n '' ']'
+ DTBFILE=tegra210-p3448-0003-p3542-0000.dtb
+ '[' -f cvm.bin.a5gxF ']'
+ rm -f cvm.bin.a5gxF
+ '[' -n '' ']'
+ fuselevel=fuselevel_production
+ '[' -n '' ']'
+ BOOTDEV=mmcblk0p1
+ rm -f jetson-nano-2gb-devkit_bootblob_ver.txt
+ echo NV3
+ . bsp_version
++ BSP_BRANCH=32
++ BSP_MAJOR=5
++ BSP_MINOR=1
++ BSP_VERSION=32.5.1
+ echo '# R32 , REVISION: 5.1'
+ echo 'BOARDID=3448 BOARDSKU=0000 FAB=400'
+ date +%Y%m%d%H%M%S
++ cut '-d ' -f2
++ cksum jetson-nano-2gb-devkit_bootblob_ver.txt
+ bytes=76
++ cut '-d ' -f1
++ cksum jetson-nano-2gb-devkit_bootblob_ver.txt
+ cksum=4167525033
+ echo 'BYTES:76 CRC32:4167525033'
+ '[' -z yes ']'
+ appfile=final-project-image.ext4
+ datafile=
+ appfile_sed=
+ '[' 0 -ne 0 ']'
+ '[' 0 -eq 0 -a -z yes ']'
+ pre_sdcard_sed=-es,APPFILE,final-project-image.ext4,
+ '[' -n '' ']'
+ touch APPFILE
+ '[' '' = yes ']'
+ cp flash.xml.in flash.xml.tmp
+ sed -es,VERFILE,jetson-nano-2gb-devkit_bootblob_ver.txt, -es,DTBFILE,tegra210-p3448-0003-p3542-0000.dtb, flash.xml.tmp
+ rm flash.xml.tmp
+ boardcfg=
+ '[' -z '' ']'
+ '[' 0 -ne 0 -o yes = yes ']'
+ cmd=sign
+ binargs=
+ '[' -n '' ']'
+ flashcmd='python3 /home/user/build/sdprep/tegraflash.py  --bl cboot.bin --bct "jetson-nano-2gb-devkit.cfg" --odmdata 0xA4000  --bldtb "tegra210-p3448-0003-p3542-0000.dtb" --applet nvtboot_recovery.bin   --cfg flash.xml --chip 0x21 --cmd "sign" '
+ '[' 0 -ne 0 ']'
+ '[' -z '' ']'
+ eval 'python3 /home/user/build/sdprep/tegraflash.py  --bl cboot.bin --bct "jetson-nano-2gb-devkit.cfg" --odmdata 0xA4000  --bldtb "tegra210-p3448-0003-p3542-0000.dtb" --applet nvtboot_recovery.bin   --cfg flash.xml --chip 0x21 --cmd "sign" '
++ python3 /home/user/build/sdprep/tegraflash.py --bl cboot.bin --bct jetson-nano-2gb-devkit.cfg --odmdata 0xA4000 --bldtb tegra210-p3448-0003-p3542-0000.dtb --applet nvtboot_recovery.bin --cfg flash.xml --chip 0x21 --cmd sign
Welcome to Tegra Flash
version 1.0.0
Type ? or help for help and q or quit to exit
Use ! to execute system commands
 
[   0.1645 ] tegrasign --key None --getmode mode.txt
[   0.1719 ] Assuming zero filled SBK key
[   0.4243 ] 
[   0.5412 ] Generating RCM messages
[   0.5777 ] tegrarcm --listrcm rcm_list.xml --chip 0x21 0 --download rcm nvtboot_recovery.bin 0 0
[   0.5903 ] RCM 0 is saved as rcm_0.rcm
[   0.6168 ] RCM 1 is saved as rcm_1.rcm
[   0.6168 ] List of rcm files are saved in rcm_list.xml
[   0.6168 ] 
[   0.6169 ] Signing RCM messages
[   0.6194 ] tegrasign --key None --list rcm_list.xml --pubkeyhash pub_key.key
[   0.6375 ] Assuming zero filled SBK key
[   0.6391 ] 
[   0.6391 ] Copying signature to RCM mesages
[   0.6410 ] tegrarcm --chip 0x21 0 --updatesig rcm_list_signed.xml
[   0.6501 ] 
[   0.6504 ] Parsing partition layout
[   0.7122 ] tegraparser --pt flash.xml.tmp
[   0.8749 ] 
[   0.8751 ] Creating list of images to be signed
[   1.0182 ] tegrahost --chip 0x21 0 --partitionlayout flash.xml.bin --list images_list.xml
[   1.2938 ] 
[   1.2942 ] Generating signatures
[   1.2973 ] tegrasign --key None --list images_list.xml --pubkeyhash pub_key.key
[   1.3020 ] Assuming zero filled SBK key
[   1.4597 ] 
[   1.4598 ] Generating br-bct
[   1.4925 ] tegrabct --bct jetson-nano-2gb-devkit.cfg --chip 0x21 0
[   1.4982 ] Copying Sdram info from 2 to 3 set
[   1.6015 ] 
[   1.6018 ] Updating boot device parameters
[   1.6057 ] tegrabct --bct jetson-nano-2gb-devkit.bct --chip 0x21 0 --updatedevparam flash.xml.bin
[   1.6081 ] Warning: No sdram params
[   1.6089 ] 
[   1.6089 ] Updating bl info
[   1.6118 ] tegrabct --bct jetson-nano-2gb-devkit.bct --chip 0x21 0 --updateblinfo flash.xml.bin --updatesig images_list_signed.xml
[   1.6184 ] 
[   1.6185 ] Updating secondary storage information into bct
[   1.6208 ] tegraparser --pt flash.xml.bin --chip 0x21 0 --updatecustinfo jetson-nano-2gb-devkit.bct
[   1.6815 ] 
[   1.6819 ] Updating Odmdata
[   1.6845 ] tegrabct --bct jetson-nano-2gb-devkit.bct --chip 0x21 0 --updatefields Odmdata =0xA4000
[   1.6956 ] Warning: No sdram params
[   1.6957 ] 
[   1.6958 ] Get Signed section of bct
[   1.7018 ] tegrabct --bct jetson-nano-2gb-devkit.bct --chip 0x21 0 --listbct bct_list.xml
[   1.7173 ] 
[   1.7174 ] Signing BCT
[   1.7302 ] tegrasign --key None --list bct_list.xml --pubkeyhash pub_key.key
[   1.7371 ] Assuming zero filled SBK key
[   1.7372 ] 
[   1.7373 ] Updating BCT with signature
[   1.7481 ] tegrabct --bct jetson-nano-2gb-devkit.bct --chip 0x21 0 --updatesig bct_list_signed.xml
[   1.7576 ] 
[   1.7578 ] Copying signatures
[   1.7612 ] tegrahost --chip 0x21 0 --partitionlayout flash.xml.bin --updatesig images_list_signed.xml
[   1.8065 ] 
[   1.8068 ] Updating BFS information on BCT
[   1.8116 ] tegrabct --bct jetson-nano-2gb-devkit.bct --chip 0x21 0 --updatebfsinfo flash.xml.bin
[   1.8267 ] 
[   1.8268 ] Copying signed file in /home/user/build/sdprep/signed
+ '[' -n yes ']'
+ '[' -n -es,APPFILE,final-project-image.ext4, ']'
+ rm -f signed/flash.xml.in
+ mv signed/flash.xml signed/flash.xml.in
+ sed -es,APPFILE,final-project-image.ext4, signed/flash.xml.in
+ /home/user/build/sdprep/make-sdcard -s 16G -b final-project-image signed/flash.xml ../test.img
Init...ERR: could not initialize GPT on ../test.img

@CalebProvost
Copy link
Contributor Author

It appears docker was missing a dependency to run the ./dosdcard.sh scripts and none of the tools checked before-hand to validate they were installed or not.

Updating docker container to have dependency gdisk is the fix to this bug.

@CalebProvost
Copy link
Contributor Author

Installed the missing dependency to the docker container and pushed the changes to docker hub.
I've updated the instructions to reflect the changes ... which btw makes things easier for end-users.

Closing as blocker is resolved.

@CalebProvost CalebProvost added complete and removed idea/discussion Not work to be done; but might be labels Apr 18, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant