Breakpoints not working for gdb while debugging remote arm target on qemu

Issue

I have compiled this baremetal example : https://github.com/s-matyukevich/raspberry-pi-os/tree/master/src/lesson01. I modified the Makefile to have debug symbols like below :

diff --git a/src/lesson01/Makefile b/src/lesson01/Makefile
index 4f92a49..daa1f7d 100644
--- a/src/lesson01/Makefile
+++ b/src/lesson01/Makefile
@@ -1,7 +1,7 @@
-ARMGNU ?= aarch64-linux-gnu
+ARMGNU ?= /opt/gcc-arm-10.3-2021.07-x86_64-aarch64-none-linux-gnu/bin/aarch64-none-linux-gnu

-COPS = -Wall -nostdlib -nostartfiles -ffreestanding -Iinclude -mgeneral-regs-only
-ASMOPS = -Iinclude
+COPS = -Wall -nostdlib -nostartfiles -ffreestanding -Iinclude -mgeneral-regs-only -g
+ASMOPS = -Iinclude -g

 BUILD_DIR = build
 SRC_DIR = src
@@ -27,5 +27,5 @@ DEP_FILES = $(OBJ_FILES:%.o=%.d)
 -include $(DEP_FILES)

 kernel8.img: $(SRC_DIR)/linker.ld $(OBJ_FILES)
-       $(ARMGNU)-ld -T $(SRC_DIR)/linker.ld -o $(BUILD_DIR)/kernel8.elf  $(OBJ_FILES)
+       $(ARMGNU)-ld -g -T $(SRC_DIR)/linker.ld -o $(BUILD_DIR)/kernel8.elf  $(OBJ_FILES)
        $(ARMGNU)-objcopy $(BUILD_DIR)/kernel8.elf -O binary kernel8.img

From one terminal, I run the program on qemu :

$ qemu-system-aarch64 -M raspi3 -kernel kernel8.img -display none -serial null -serial stdio -S -s

From other terminal, I launch the gdb :

$ /opt/gcc-arm-10.3-2021.07-x86_64-aarch64-none-linux-gnu/bin/aarch64-none-linux-gnu-gdb ./build/kernel8.elf -ex 'target remote localhost:1234' -ex 'break _start' -ex 'break kernel_main' -ex 'continue'

But the breakpoints are never hit.

GNU gdb (GNU Toolchain for the A-profile Architecture 10.3-2021.07 (arm-10.29)) 10.2.90.20210621-git
Copyright (C) 2021 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "--host=x86_64-pc-linux-gnu --target=aarch64-none-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://bugs.linaro.org/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from ./build/kernel8.elf...
Remote debugging using localhost:1234
_start () at src/boot.S:7
7       mrs x0, mpidr_el1
Breakpoint 1 at 0x0: file src/boot.S, line 7.
Breakpoint 2 at 0x234: file src/kernel.c, line 5.
Continuing.

What am I doing wrong? I have looked all related questions, but nothing is helping. Here are my images for reference : kernel8.elf and kernel8.img

EDIT

I do get Hello, world! on my console when I continue, so the kernel8.img is booting fine. Also, for reference :

$ qemu-system-aarch64 --version
QEMU emulator version 4.2.1 (Debian 1:4.2-3ubuntu6.18)
Copyright (c) 2003-2019 Fabrice Bellard and the QEMU Project developers

Solution

GDB is placing breakpoints based on where the ELF file says the code is. You can see that in your transcript it thinks the _start function in boot.S is at address 0x0. However, when you tell QEMU to load your binary file, you are not doing that in a way that matches what the ELF file says. So the actual code being executed is at an entirely different address, and the breakpoints are not in addresses that match the executing code, so they don't hit. Since you're not compiling the code to be position-independent, when it runs from this address it is working mostly by luck (because even non-position-independent aarch64 code often doesn't have position-dependent instructions in it).

The reason the addresses don't match up is because your ELF file is saying code starts at address 0x0, but you're passing QEMU a binary file to the -kernel option, which means "I am a Linux kernel, boot me the way the Linux kernel boot protocol says to do that" (see https://www.kernel.org/doc/Documentation/arm64/booting.txt). This means a number of things, including that (for the current QEMU implementation -- this isn't strictly mandated by the booting protocol) we load the image to the address 0x80000, and run a bit of stub code generated by QEMU which sets up some registers and jumps to that location. That is why when you set your linker script to link the image to that address it happens to start working.

The solution to this is to make a choice about how you want to boot your guest code:

• You can make it honour the various requirements of the Linux kernel boot protocol, and pass it as a binary file to -kernel
• You can write it to be a pure bare-metal image that includes a vector table at address 0x0, and load it with the QEMU "generic loader", which will take an ELF file and load all its segments as the ELF headers specify. (It is also possible to pass an ELF file to -kernel, but the generic loader makes more sense in this situation.)

QEMU does not support "load this ELF file and start it in the way that the Raspberry Pi firmware supports running an ELF file that it loads from an SD card". So you may need to make some adjustments to bare-metal code tutorials that were designed only for running on real hardware.

(For more info on the various QEMU options for loading guest code, see this answer.)

Answered By - Peter Maydell