What and how to analyze a hung process in Linux (mono, CPU 100%)?

Question

Mono

Hello!
In general, briefly and to the point:
There is some client-server application written in C ++
The server part (.exe) runs under mono in linux
Periodically, for no apparent reason, the server application under mono consumes all the CPU.
We need to understand, to get at least some information about what exactly is looping, what is happening, for further investigation and fixing this problem.
In general, I would like to know about the following:

1. Who uses what means to search for at least some information about what is happening in a hung process by PID
   
2. What applications and why?
   
3. Can you show examples?
   
4. What other tools are there to analyze what the application does when it freezes (in terms of network, in terms of disk usage, in terms of what calls are called, etc.)
   

Answer 1

1. gdb, strace
2. ?
3. Examples: gdb:

$ pidof top
21259
$ gdb
GNU gdb (GDB) Fedora 7.5.1-42.fc18
...
(gdb) attach 21259
Attaching to process 21259
Reading symbols from /usr/bin/top...Reading symbols from /usr/bin/top...(no debugging symbols found)...done.
...
(gdb) bt
#0  0x0000003fd2ceb843 in __select_nocancel () from /lib64/libc.so.6
#1  0x00000000004033e7 in main ()
(gdb) up
#1  0x00000000004033e7 in main ()
(gdb) x/10i $pc
=> 0x4033e7 <main+3639>:        test   %eax,%eax
   0x4033e9 <main+3641>:        jle    0x403340 <main+3472>
   0x4033ef <main+3647>:        xor    %edi,%edi
   0x4033f1 <main+3649>:        callq  0x408840 <keyin>
   0x4033f6 <main+3654>:        cmp    $0x1b,%eax
   0x4033f9 <main+3657>:        mov    %eax,%ebx
   0x4033fb <main+3659>:        je     0x403340 <main+3472>
   0x403401 <main+3665>:        jg     0x40344e <main+3742>
   0x403403 <main+3667>:        test   %eax,%eax
   0x403405 <main+3669>:        je     0x403340 <main+3472>
(gdb)

It will be much prettier if the application has debug information.
strace:

$ strace -i -p 21259
Process 21259 attached
[      3fd2ceb843] select(1, [0], NULL, NULL, {2, 80709}) = 0 (Timeout)
...

-- see "what calls are being called".