From: Michael Olbrich <m.olbrich@pengutronix.de>
To: Roland Hieber <rhi@pengutronix.de>, ptxdist@pengutronix.de
Subject: Re: [ptxdist] [PATCH v2] doc: daily work: update debugging section
Date: Thu, 19 Sep 2024 11:34:01 +0200 [thread overview]
Message-ID: <ZuvwCbwXnb82OuJU@pengutronix.de> (raw)
In-Reply-To: <20240826120418.2528432-1-rhi@pengutronix.de>
On Mon, Aug 26, 2024 at 02:04:18PM +0200, Roland Hieber wrote:
> * Split off the "Running QEMU" section into a stand-alone section
> * Mention /usr/lib/debug instead of the old ./debug/ folders
> * Describe `ptxdist gdb` and debugging on the target directly
> * Keep GDB usage short; refer to third-party documentation
> * Use literal blocks instead of code blocks without highlighting
> * Make use of the |ptxdistPlatformDir| template variable
>
> Signed-off-by: Roland Hieber <rhi@pengutronix.de>
> ---
> PATCH v2:
> * somehow I forgot to stage & amend various changes before sending v1,
> please disregard the old one
>
>
> doc/daily_work.inc | 226 ++++++++++++++++++++++++++++++++++-----------
> 1 file changed, 174 insertions(+), 52 deletions(-)
>
> diff --git a/doc/daily_work.inc b/doc/daily_work.inc
> index b98adf75197d..65b0bad25d97 100644
> --- a/doc/daily_work.inc
> +++ b/doc/daily_work.inc
> @@ -285,15 +285,11 @@ nonexisting file, we can limit the output to all ``open`` system calls:
> The binary may fail due to a missing ``/etc/foo.conf``. This could be a
> hint on what is going wrong (it might not be the final solution).
>
> -Debugging with CPU emulation
> -----------------------------
> -
> -If we do not need some target related feature to run our application, we
> -can also debug it through a simple CPU emulation. Thanks to QEMU we can
> -run ELF binaries for other architectures than our build host is.
> +Running an Application Made for a Different Architecture
> +--------------------------------------------------------
>
> -Running an Application made for a different Architecture
> -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> +Thanks to QEMU we can run ELF binaries for different architectures than our
> +build host.
>
> PTXdist creates a fully working root filesystem with all run-time
> components in ``root/``. Lets assume we made a PTXdist based project for
> @@ -318,64 +314,134 @@ host’s operating system. Using QEMU in this way let us simply check our
> programs. There are also QEMU environments for other architectures
> available.
>
> -Debugging an Application made for a different Architecture
> -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> -
> -Debugging our application is also possible with QEMU. All we need are a
> -root filesystem with debug symbols available, QEMU and an architecture
> -aware debugger.
> -
> -The root filesystem with debug symbols will be provided by PTXdist, the
> -architecture aware debugger comes with the OSELAS.Toolchain. Two
> -consoles are required for this debug session in this example. We start
> -the QEMU in the first console as:
> -
> -.. code-block:: text
> -
> - $ cd ptxdistPlatformDir/root
> - ptxdistPlatformDir/root$ qemu-<architecture> -g 1234 -cpu <cpu-core> -L . usr/bin/myapp
> +Debugging an Application Made for a Different Architecture
> +----------------------------------------------------------
> +
> +Debugging an application is possible in three ways:
> +
> +#. :ref:`Directly running GDB on the target hardware <gdb_on_target>`:
> + This approach has the advantage that the target system is "self-contained",
> + but it is necessary to install the debug symbols in the target's root file
> + system, which can significantly improve the memory footprint. Furthermore,
"improve" sounds wrong here. Maybe you meant "increase"?
> + it is often not possible for GDB to access the application's source code
> + unless further steps are taken (e.g. making the source code available in the
> + correct paths over an NFS share or from a USB drive).
> +
> +#. :ref:`Running the application in a QEMU emulating the target architecture <gdb_via_qemu>`:
> + This approach doesn't require any additional hardware and can therefore be
> + done completely on the build host, but it is limited to applications
> + that do not depend on specific behaviour of the target hardware, like
> + using external interfaces or architecture-specific features that are not
> + available via QEMU (e.g. accessing GPS hardware over UART, or graphics
> + acceleration).
> +
> +#. :ref:`Running gdbserver on the target hardware, and connecting to it with GDB
> + from the build host <ptxdist_gdb>`:
> + This is the simplest use case when a target hardware with network interfaces
> + is available, and PTXdist includes native support for it.
> + In this case, the target file system only has to include the *gdbserver*
> + binary, but does not need to contain any further debug information or source
> + code, which are supplied by the build host.
> +
> +In any case, we need the respective binaries including their debug information,
> +and a debugger for the target architecture.
> +The root filesystem with debug symbols is provided by PTXdist in
> +``|ptxdistPlatformDir|/root/`` on the build host, and the architecture-aware debugger
> +comes with the OSELAS.Toolchain.
> +
> +.. note:: When PTXdist builds a target root file system, the binaries that are
> + installed into it during the *targetinstall* stage of each package are
> + always stripped of their debug symbols. Instead, the debug symbols are
> + installed under ``|ptxdistPlatformDir|/root/usr/lib/debug/`` in separate files named
> + after the *build ID* of the binary (which is a checksum over the contents of
> + the binary, and can change when a binary is rebuilt).
> +
> + When GDB loads a binary, it will look for and automatically load matching
> + debug symbols in ``usr/lib/debug`` relative to its *sysroot*, which is
> + usually ``/``, but can be configured at runtime.
> +
> +.. note:: Information on using GDB is not part of this manual.
> + There are countless cheatsheets available on the internet, but in most
> + cases you will get around sufficiently with the ``break <location|function>``,
> + ``run``, ``next``, ``step``, ``finish``, ``continue``, ``print <statement|address>``,
> + ``list <location|function>`` and ``quit`` commands.
> + Current GDB versions also include a well-structured online help with the
> + ``help`` command, which can show you how to use each command.
> +
> +.. _gdb_on_target:
> +
> +Running GDB on the Target Hardware
> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> +
> +In this case, GDB and the debug information need to be available on the target hardware.
> +GDB can be enabled in the *menuconfig* with the ``PTXCONF_GDB`` symbol.
> +Additionally, the debug symbols are installed into the target root file system
> +when the ``PTXCONF_DEBUG_PACKAGES_INSTALL`` option is enabled.
> +(If an image has been built before, i.e. the *targetinstall* stages have
> +already been executed, a ``ptxdist clean root`` followed by a ``ptxdist go`` or
> +``ptxdist images`` might be needed to install all debug symbols.)
> +
> +Due to the size of the resulting root file system, the target root file system
> +is often supplied via :ref:`NFSROOT <nfsroot>`.
> +
> +On the target, GDB can then be used normally (with the caveat that the source files are
> +not available unless supplied manually)::
> +
> + root@target $ gdb /usr/bin/systemctl
> + Reading symbols from /usr/bin/systemctl...
> + Reading symbols from /usr/lib/debug/.build-id/fb/3b770e637b51276dcdc41a1e8b37df8a6d6686.debug...
> + (gdb) break main
> + Breakpoint 1 at 0x6c90: file ../systemd-256.1/src/systemctl/systemctl.c, line 1372.
> + (gdb) run
> + Starting program: /usr/bin/systemctl
> + [Thread debugging using libthread_db enabled]
> + Using host libthread_db library "/lib/libthread_db.so.1".
> +
> + Breakpoint 1, main (argc=1, argv=0xbee9bd64)
> + at ../systemd-256.1/src/systemctl/systemctl.c:1372
> + 1372 in ../systemd-256.1/src/systemctl/systemctl.c
> + (gdb) backtrace
> + #0 main (argc=1, argv=0xbee9bd64)
> + at ../systemd-256.1/src/systemctl/systemctl.c:1372
> + (gdb) list
> + 1372 ../systemd-256.1/src/systemctl/systemctl.c: No such file or directory.
> + (gdb) quit
> +
> +.. _gdb_via_qemu:
> +
> +Debugging with CPU Emulation
> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> +
> +If we do not need target-specific features to run our application, we can also
> +debug it through CPU emulation.
> +Two consoles are required for this debug session in this example. We start
> +the QEMU in the first console as::
>
> -.. note:: PTXdist always builds a root filesystem ``root/``.
> - It contains all components without debug
> - information (all binaries are in the same size as used later on on the
> - real target). In addition, each directory that contains binaries also
> - contains a ``.debug/`` directory. It contains a file with only the debug
> - symbols for each binary. These files are ignored while running
> - applications but GDB knows about it and will automatically load the debug
> - files.
> + $ cd |ptxdistPlatformDir|/root
> + |ptxdistPlatformDir|/root$ qemu-<architecture> -g 1234 -cpu <cpu-core> -L . usr/bin/myapp
>
> The added *-g 1234* parameter lets QEMU wait for a GDB connection to run
> the application.
>
> In the second console we start GDB with the correct architecture
> support. This GDB comes with the same OSELAS.Toolchain that was also
> -used to build the project:
> -
> -.. code-block:: text
> +used to build the project::
>
> - $ ./selected_toolchain/<target>-gdb --tui platform-<platformname>/root/usr/bin/myapp
> -
> -This will run a *curses* based GDB. Not so easy to handle (we must enter
> -all the commands and cannot click with a mouse!), but very fast to take
> -a quick look at our application.
> + $ ./selected_toolchain/<target>-gdb |ptxdistPlatformDir|/root/usr/bin/myapp
>
> At first we tell GDB where to look for debug symbols. The correct
> -directory here is ``root/``.
> -
> -.. code-block:: text
> +directory here is ``|ptxdistPlatformDir|/root/``::
>
> - (gdb) set solib-absolute-prefix platform-<platformname>/root
> + (gdb) set solib-absolute-prefix |ptxdistPlatformDir|/root
This part of the documentation is pretty old and a bit outdated. Can you
change it to use 'ptxdist gdb'? It should possible to use it in this case
and then manually setting 'solib-absolute-prefix' is not needed.
Michael
>
> -Next we connect this GDB to the waiting QEMU:
> -
> -.. code-block:: text
> +Next we connect this GDB to the waiting QEMU::
>
> (gdb) target remote localhost:1234
> Remote debugging using localhost:1234
> [New Thread 1]
> 0x40096a7c in _start () from root/lib/ld.so.1
>
> -As our application is already started, we can’t use the GDB command
> +As our application is already started via QEMU, we can’t use the GDB command
> ``start`` to run it until it reaches ``main()``. We set a breakpoint
> instead at ``main()`` and *continue* the application:
>
> @@ -387,10 +453,6 @@ instead at ``main()`` and *continue* the application:
> Continuing.
> Breakpoint 1, main (argc=1, argv=0x4007f03c) at myapp.c:644
>
> -The top part of the running gdbtui console will always show us the
> -current source line. Due to the ``root/`` directory usage all
> -debug information for GDB is available.
> -
> Now we can step through our application by using the commands *step*,
> *next*, *stepi*, *nexti*, *until* and so on.
>
> @@ -403,6 +465,66 @@ Now we can step through our application by using the commands *step*,
> debug symbols kept, it will be also possible for GDB to debug C library
> functions our application calls (so it might worth the disk space).
>
> +.. _ptxdist_gdb:
> +
> +Remote-Debugging with *gdbserver*
> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> +
> +When debugging with *gdbserver*, the debugging session is split into two parts:
> +the server runs on the target and is remote-controlled by a GDB instance
> +running on the build host over a network connection.
> +You need to enable the ``PTXCONF_GDBSERVER`` symbol in PTXdists's *menuconfig*
> +which makes *gdbserver* available on your target.
> +
> +Like for the approach above, you will need two terminals.
> +In the first terminal you connect to your running target (e.g. via SSH or
> +serial console), start *gdbserver* and tell it the binary to debug and where
> +to listen for connections::
> +
> + root@target $ gdbserver 192.168.1.11:1234 /usr/bin/systemctl
> + Process /usr/bin/systemctl created; pid = 477
> + Listening on port 1234
> +
> +In the above example, 192.168.1.11 is the outside-facing IP address of the
> +target, and 1234 is the port on which to listen.
> +gdbserver has now created a process, and waits for connections from a GDB.
> +
> +In the other terminal, you then start ``ptxdist gdb``, which will set up GDB
> +correctly so that it can find all necessary debug symbols in PTXdist's
> +``|ptxdistPlatformDir|/root/`` folder, and the sources in
> +``|ptxdistPlatformDir|/sysroot-target/``.
> +You then connect to the gdbserver instance running on the target using GDB's
> +*target remote* command::
> +
> + user@buildhost $ ptxdist gdb
> + (gdb) target remote 192.168.9.10:1234
> + Remote debugging using 192.168.9.10:1234
> + Reading symbols from …/my-bsp/|ptxdistPlatformDir|/root/usr/bin/systemctl...
> + Reading symbols from …/my-bsp/|ptxdistPlatformDir|/root/usr/lib/debug/.build-id/fb/3b770e637b51276dcdc41a1e8b37df8a6d6686.debug...
> + Reading symbols from …/my-bsp/|ptxdistPlatformDir|/root/lib/ld-linux-armhf.so.3...
> + Reading symbols from …/my-bsp/|ptxdistPlatformDir|/root/usr/lib/debug/.build-id/0d/a8d3736dcd31fbf43ea0cb79d1f9699c6695ff.debug...
> + 0xb6f29624 in _start () from …/my-bsp/|ptxdistPlatformDir|/root/lib/ld-linux-armhf.so.3
> + (gdb) break main
> + Breakpoint 1 at 0x4e3c90: file ../systemd-256.1/src/systemctl/systemctl.c, line 1372.
> + (gdb) continue
> + Continuing.
> +
> + Breakpoint 1, main (argc=1, argv=0xbe829d84) at ../systemd-256.1/src/systemctl/systemctl.c:1372
> + 1372 DEFINE_MAIN_FUNCTION_WITH_POSITIVE_FAILURE(run);
> + (gdb) list
> + 1367
> + 1368 /* Note that we return r here, not 0, so that we can implement the LSB-like return codes */
> + 1369 return r;
> + 1370 }
> + 1371
> + 1372 DEFINE_MAIN_FUNCTION_WITH_POSITIVE_FAILURE(run);
> + 1373 #endif
> + (gdb)
> +
> +You can now use GDB as usual, and when the debugged process terminates (or you
> +quit the GDB session), *gdbserver* will terminate as well, and close the
> +connection.
> +
> Migration between Releases
> --------------------------
>
> --
> 2.39.2
>
>
>
--
Pengutronix e.K. | |
Steuerwalder Str. 21 | http://www.pengutronix.de/ |
31137 Hildesheim, Germany | Phone: +49-5121-206917-0 |
Amtsgericht Hildesheim, HRA 2686 | Fax: +49-5121-206917-5555 |
prev parent reply other threads:[~2024-09-19 9:34 UTC|newest]
Thread overview: 3+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-08-23 15:26 [ptxdist] [PATCH] " Roland Hieber
2024-08-26 12:04 ` [ptxdist] [PATCH v2] " Roland Hieber
2024-09-19 9:34 ` Michael Olbrich [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=ZuvwCbwXnb82OuJU@pengutronix.de \
--to=m.olbrich@pengutronix.de \
--cc=ptxdist@pengutronix.de \
--cc=rhi@pengutronix.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox