Debugging

From Maemo Leste Wiki
Revision as of 13:27, 5 May 2021 by Wizzup (talk | contribs) (Created page with "=== Things break. How do you debug the problems? === You will want to take the following steps: * Add the devuan/debian debug repositories. * Identify the program that crash...")
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigationJump to search

Things break. How do you debug the problems?

You will want to take the following steps:

  • Add the devuan/debian debug repositories.
  • Identify the program that crashes (you can use `dpkg -S /usr/bin/program-name`) and install the -dbg or -dbgsym packages
  • Install the -dbg and -dbgsym dependencies of the packages
  • Attach to the program in gdb (or start it from gdb)


Add the debug repositories

Maemo Leste ships its debug packages in the normal repository, but for the Devuan/Debian packages, you need to add an additional source list file: 

# cat /etc/apt/sources.list.d/debug.list
deb http://deb.debian.org/debian-debug/ buster-debug main
deb http://deb.debian.org/debian-debug/ buster-backports-debug main
deb https://pkgmaster.devuan.org/devuan beowulf main contrib non-free
deb https://pkgmaster.devuan.org/devuan beowulf-updates main contrib non-free
deb https://pkgmaster.devuan.org/devuan beowulf-security main contrib non-free


Identify the program that crashes

Example: location-status (the location status applet) crashes.

Find out which main program or library is likely involved in the crash. If clicking the location status applet button causes a crash, the problem is likely in that package, and we can use dpkg -S to figure out what package the file is part of, and then install the debug symbols: 

# dpkg -S /usr/lib/arm-linux-gnueabihf/hildon-desktop/location-status.so
location-status: /usr/lib/arm-linux-gnueabihf/hildon-desktop/location-status.so
# apt-cache search location-status | grep dbg
location-status-dbgsym - debug symbols for location-status
# apt install location-status-dbgsym
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following package was automatically installed and is no longer required:
  libmicrodns0
Use 'apt autoremove' to remove it.
The following NEW packages will be installed:
  location-status-dbgsym
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 27,1 kB of archives.
After this operation, 41,0 kB of additional disk space will be used.
0% [Working]
Get:1 https://maedevu.maemo.org/leste beowulf/main armhf location-status-dbgsym armhf 0.111-1+2m7 [27,1 kB]
Fetched 27,1 kB in 0s (55,8 kB/s)
Selecting previously unselected package location-status-dbgsym.
(Reading database ... 96891 files and directories currently installed.)
Preparing to unpack .../location-status-dbgsym_0.111-1+2m7_armhf.deb ...
Unpacking location-status-dbgsym (0.111-1+2m7) ...
Setting up location-status-dbgsym (0.111-1+2m7) ...

Install additional debug files

To get useful stack traces, gdb typically requires the debug symbols of other libraries to be installed as well. Following our example location-status, we can get a list of all the libraries involved using ldd: 

# ldd /usr/lib/arm-linux-gnueabihf/hildon-desktop/location-status.so

This lists a lot of libraries, so we will start with the basic ones, e.g. glib-2.0: 

root@maindroid:/etc/apt# dpkg -S /usr/lib/arm-linux-gnueabihf/libglib-2.0.so.0.5800.3
libglib2.0-0:armhf: /usr/lib/arm-linux-gnueabihf/libglib-2.0.so.0.5800.3
root@maindroid:/etc/apt# apt-cache search libglib2.0-0 | grep dbg
libglib2.0-0-dbgsym - debug symbols for libglib2.0-0
# apt install libglib2.0-0-dbgsym
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following package was automatically installed and is no longer required:
  libmicrodns0
Use 'apt autoremove' to remove it.
The following NEW packages will be installed:
  libglib2.0-0-dbgsym
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 4240 kB of archives.
After this operation, 4984 kB of additional disk space will be used.
Get:1 http://deb.debian.org/debian-debug buster-debug/main armhf libglib2.0-0-dbgsym armhf 2.58.3-2+deb10u2 [4240 kB]
0% [1 libglib2.0-0-dbgsym 0 B/4240 kB 0%]
Fetched 4240 kB in 3s (1278 kB/s)
Selecting previously unselected package libglib2.0-0-dbgsym:armhf.
(Reading database ... 96893 files and directories currently installed.)
Preparing to unpack .../libglib2.0-0-dbgsym_2.58.3-2+deb10u2_armhf.deb ...
Unpacking libglib2.0-0-dbgsym:armhf (2.58.3-2+deb10u2) ...
Setting up libglib2.0-0-dbgsym:armhf (2.58.3-2+deb10u2) ...

We can repeat this process until we get satisfactory backtraces in gdb. For example: 

# gdb attach <pid>
[...]
Thread 1 "hildon-status-m" received signal SIGSEGV, Segmentation fault.
0xb6625f72 in g_atomic_ref_count_compare (arc=arc@entry=0x38, val=val@entry=0) at ../../../glib/grefcount.c:284
284	../../../glib/grefcount.c: No such file or directory.
(gdb) bt
#0  0xb6625f72 in g_atomic_ref_count_compare (arc=arc@entry=0x38, val=val@entry=0) at ../../../glib/grefcount.c:284
#1  0xb660cf0a in g_hash_table_lookup_node
    (hash_return=<synthetic pointer>, key=0xbed97c68, hash_table=<error reading variable: Cannot access memory at address 0x30>0x10) at ../../../glib/ghash.c:1153
#2  0xb660cf0a in g_hash_table_lookup (hash_table=<error reading variable: Cannot access memory at address 0x30>0x10, key=0xbed97c68) at ../../../glib/ghash.c:1153
#3  0xb64b03fe in  () at /usr/lib/arm-linux-gnueabihf/libosso.so.1

This means we should probably install debug symbols for libosso.so.1, so we add it and repeat: 

root@maindroid:/etc/apt# dpkg -S /usr/lib/arm-linux-gnueabihf/libosso.so.1
libosso1: /usr/lib/arm-linux-gnueabihf/libosso.so.1
root@maindroid:/etc/apt# apt-cache search libosso1 | grep dbg
libosso1-dbgsym - debug symbols for libosso1
root@maindroid:/etc/apt# apt install libosso1-dbgsym
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following package was automatically installed and is no longer required:
  libmicrodns0
Use 'apt autoremove' to remove it.
The following NEW packages will be installed:
  libosso1-dbgsym
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 111 kB of archives.
After this operation, 137 kB of additional disk space will be used.
Get:1 https://maedevu.maemo.org/leste beowulf/main armhf libosso1-dbgsym armhf 2.35+2m7 [111 kB]
Fetched 111 kB in 1s (197 kB/s)
Selecting previously unselected package libosso1-dbgsym.
(Reading database ... 96903 files and directories currently installed.)
Preparing to unpack .../libosso1-dbgsym_2.35+2m7_armhf.deb ...
Unpacking libosso1-dbgsym (2.35+2m7) ...
Setting up libosso1-dbgsym (2.35+2m7) ...

Try again in gdb: 

[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/arm-linux-gnueabihf/libthread_db.so.1".
__libc_do_syscall () at ../sysdeps/unix/sysv/linux/arm/libc-do-syscall.S:46
46	../sysdeps/unix/sysv/linux/arm/libc-do-syscall.S: No such file or directory.
(gdb) c
Continuing.

Thread 1 "hildon-status-m" received signal SIGSEGV, Segmentation fault.
0x10460f44 in ?? ()
(gdb) bt
#0  0x10460f44 in  ()
#1  0xb660cf16 in g_hash_table_lookup_node
    (hash_return=<synthetic pointer>, key=0xbed97c60, hash_table=0xb66cb765<error reading variable: Cannot access memory at address 0x2dbf004b>)
    at ../../../glib/ghash.c:379
#2  0xb660cf16 in g_hash_table_lookup (hash_table=0xb66cb765<error reading variable: Cannot access memory at address 0x2dbf004b>, key=key@entry=0xbed97c60)
    at ../../../glib/ghash.c:1153
#3  0xb64b03fe in try_plugin (dir=<optimized out>, file=file@entry=0xb474d75c "liblocation_applet.so", osso=<optimized out>) at osso-cp-plugin.c:103
#4  0xb64b0466 in osso_cp_plugin_execute (osso=0x63a4b8, filename=0xb474d75c "liblocation_applet.so", data=data@entry=0x669850, user_activated=user_activated@entry=1)
    at osso-cp-plugin.c:148
#5  0xb474d418 in execute_cp_plugin (obj=0x669850) at location-status.c:84
#9  0xb66e10b2 in <emit signal ??? on instance 0x669850 [HildonButton]> (instance=<optimized out>, signal_id=<optimized out>, detail=0)
    at ../../../gobject/gsignal.c:3447
    #6  0xb66ccb7e in g_closure_invoke (closure=0x664af8, return_value=0x0, n_param_values=1, param_values=0xbed98df0, invocation_hint=0xbed98d8c)
    at ../../../gobject/gclosure.c:810
    #7  0xb66dab0a in signal_emit_unlocked_R
    (node=node@entry=0x551458, detail=0, instance=0x669850, emission_return=emission_return@entry=0x0, instance_and_params=0xbed98df0)
    at ../../../gobject/gsignal.c:3635
    #8  0xb66e0c0e in g_signal_emit_valist (instance=<optimized out>, signal_id=<optimized out>, detail=detail@entry=1, var_args=..., var_args@entry=...)
    at ../../../gobject/gsignal.c:3391
#10 0xb6a9cd6c in  () at /usr/lib/arm-linux-gnueabihf/libgtk-x11-2.0.so.0

Do the same for libgtk-x11-2.0.so.0, etc.

TODO

  • valgrind
  • gdb

Example debugging location-status bug:

Retrieved from "https://leste.maemo.org/index.php?title=Debugging&oldid=995"