remote debugger failing on first hit with puma

[kikonen]

Your environment

• ruby -v: 3.3.5
• rdbg -v: 1.9.2

Describe the bug
When hitting breakpoint via placing "debugger" into source code, and connecting with remote debugger, with sinatra controller and puma web server in local docker container, first time when "debugger" is hit will fail, as described below

For example,
Puma started freshly serving sinatra app in local docker container

    33|       def some_method_called_via_puma_via_sinatra_controller
=>  34|         debugger
    35|         some logic...
    37|       end
    38|
(rdbg:remote) n
# No sourcefile available for /usr/local/bundle/gems/puma-6.4.2/lib/puma/single.rb

Stop by SIGURG
(rdbg:remote) n
# it looks like it tries to step in code, but in reality context is lost
(rdbg:remote) n
[1, 7] in $(Gem)/rack-2.2.9/lib/rack/file.rb
     1| # frozen_string_literal: true
     2|
     3| require_relative 'files'
     4|
=>   5| module Rack
     6|   File = Files
     7| end
(rdbg:remote) c

Thus only way to end to proceed is to hit "continue", since control flow has jumped into some odd place, and now, after this failure when doing same request again, debugger works as espected (i.e. does not hit this "SIGURG" which seems to break thing).

Makes usage rather irritating since after every modify of code (and restart of web server thus), have to first hit debugger "cold", and failing, and hitting continue, and now it seems to be "hot", and can actually succeed.

To Reproduce

• Local docker container
• running sinatra app with puma on it
• setting up remote debugger
• adding breakpoint via placing "debugger" into code
• launching app on container
• performing some request hitting "debugger" put in code
• connect remote debugger to process
  => fails on first time, if hitting continue, and retrying request it works as expected

Expected behavior
It would not fail on first time, since it makes debugging few cases difficult.

Additional context
Issue has been there for a while, i.e. it has had this failure always for me with this debug gem

[ko1]

You mean that the "fail" is showing the "# No sourcefile available for /usr/local/bundle/gems/puma-6.4.2/lib/puma/single.rb
" message, right?

I'm not sure but /usr/local/bundle/gems/puma-6.4.2/lib/puma/single.rb is available on the container?

[kikonen]

I figured out that in order to make thing work is

1. add "debugger" statement into desired place
2. let "rerun" logic to restart app in container (thus any existing remote debugger session will be killed)
3. attach remote debugger to app in aonther shell
   => it opens debugger console
   => hit continue on it
4. Do relevant things to trigger breakpoint
   => when "debugger" is hit it activaves console session in remote debug session started in step (3)
   => it seems to work as expected

However, this sequence fails

1. add "debugger" statement into desired place
2. let "rerun" logic to restart app in container (thus any existing remote debugger session will be killed)
3. Do relevant things to trigger breakpoint
   => it tells that it's waiting for remote debugger in stdout of app
4. attach remote debugger to app in aonther shell
   => console opens
   => seems to be in right place
   => but after trying to do anything, it hits issue described in original description
   => thus does not actually work
   => hitting continue is only option possibly
5. Do relevant things to trigger breakpoint again
   => when "debugger" is hit it activaves console session in remote debug session started in step (4)
   => this time it seems to work as expected

[robacarp]

I'm able to reproduce this but without anything in docker.

The procedure is:

1. add a debugger to a controller action

$ export RUBY_DEBUG_OPEN="true"
$ export RUBY_DEBUG_LAZY="true"
$ bin/rails s

3. navigate browser to the controller action, server emits

   DEBUGGER: Debugger can attach via UNIX domain socket (/var/folders/3j/9z86x6zx41v8408k3kjv_0y00000gn/T/rdbg-501/rdbg-82033)
   DEBUGGER: wait for debugger connection...
   

4. in a new terminal, rdbg --attach

Additionally, the behavior I'm seeing on the first run is characterized by the output being from "the previous command" rather than the current command. Hopefully this demonstrates what I'm seeing:

The lines with numbers after the prompt should cause the debugger to evaluate the number and show the result (the same number). Instead, the results are off-by-one.

• input: 2, output: a partial stack trace
• input: 3, output: 2
• input: 4, output: 3
• input: 5, output: 4

Probably because of this off-by-one bug, I'm often unable to actually continue and have to pkill -QUIT the rdbg process. You can see that happened in this screenshot too. If I have to manually QUIT rdbg, I have to kill -KILL the rails server because it goes completely unresponsive -- I'm not particularly surprised by this, but it's further evidence that the debug socket is in disrepair.

---

> ruby -v
ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [arm64-darwin24]

> rdbg -v
rdbg 1.10.0

Our Gemfile.lock

GEM
  remote: https://rubygems.org/
  specs:
    actioncable (6.1.7.6)
      actionpack (= 6.1.7.6)
      activesupport (= 6.1.7.6)
      nio4r (~> 2.0)
      websocket-driver (>= 0.6.1)
    actionmailbox (6.1.7.6)
      actionpack (= 6.1.7.6)
      activejob (= 6.1.7.6)
      activerecord (= 6.1.7.6)
      activestorage (= 6.1.7.6)
      activesupport (= 6.1.7.6)
      mail (>= 2.7.1)
    actionmailer (6.1.7.6)
      actionpack (= 6.1.7.6)
      actionview (= 6.1.7.6)
      activejob (= 6.1.7.6)
      activesupport (= 6.1.7.6)
      mail (~> 2.5, >= 2.5.4)
      rails-dom-testing (~> 2.0)
    actionpack (6.1.7.6)
      actionview (= 6.1.7.6)
      activesupport (= 6.1.7.6)
      rack (~> 2.0, >= 2.0.9)
      rack-test (>= 0.6.3)
      rails-dom-testing (~> 2.0)
      rails-html-sanitizer (~> 1.0, >= 1.2.0)
    actiontext (6.1.7.6)
      actionpack (= 6.1.7.6)
      activerecord (= 6.1.7.6)
      activestorage (= 6.1.7.6)
      activesupport (= 6.1.7.6)
      nokogiri (>= 1.8.5)
    actionview (6.1.7.6)
      activesupport (= 6.1.7.6)
      builder (~> 3.1)
      erubi (~> 1.4)
      rails-dom-testing (~> 2.0)
      rails-html-sanitizer (~> 1.1, >= 1.2.0)
    activejob (6.1.7.6)
      activesupport (= 6.1.7.6)
      globalid (>= 0.3.6)
    activemodel (6.1.7.6)
      activesupport (= 6.1.7.6)
    activerecord (6.1.7.6)
      activemodel (= 6.1.7.6)
      activesupport (= 6.1.7.6)
    activestorage (6.1.7.6)
      actionpack (= 6.1.7.6)
      activejob (= 6.1.7.6)
      activerecord (= 6.1.7.6)
      activesupport (= 6.1.7.6)
      marcel (~> 1.0)
      mini_mime (>= 1.1.0)
    activesupport (6.1.7.6)
      concurrent-ruby (~> 1.0, >= 1.0.2)
      i18n (>= 1.6, < 2)
      minitest (>= 5.1)
      tzinfo (~> 2.0)
      zeitwerk (~> 2.3)
    ast (2.4.2)
    autoprefixer-rails (10.4.16.0)
      execjs (~> 2)
    awesome_print (1.9.2)
    aws-eventstream (1.3.0)
    aws-partitions (1.880.0)
    aws-sdk-autoscaling (1.102.0)
      aws-sdk-core (~> 3, >= 3.188.0)
      aws-sigv4 (~> 1.1)
    aws-sdk-core (3.190.2)
      aws-eventstream (~> 1, >= 1.3.0)
      aws-partitions (~> 1, >= 1.651.0)
      aws-sigv4 (~> 1.8)
      jmespath (~> 1, >= 1.6.1)
    aws-sdk-ec2 (1.433.0)
      aws-sdk-core (~> 3, >= 3.188.0)
      aws-sigv4 (~> 1.1)
    aws-sdk-kms (1.76.0)
      aws-sdk-core (~> 3, >= 3.188.0)
      aws-sigv4 (~> 1.1)
    aws-sdk-rds (1.211.0)
      aws-sdk-core (~> 3, >= 3.188.0)
      aws-sigv4 (~> 1.1)
    aws-sdk-s3 (1.142.0)
      aws-sdk-core (~> 3, >= 3.189.0)
      aws-sdk-kms (~> 1)
      aws-sigv4 (~> 1.8)
    aws-sigv4 (1.8.0)
      aws-eventstream (~> 1, >= 1.0.2)
    base64 (0.2.0)
    bcrypt (3.1.20)
    bigdecimal (3.1.5)
    bootsnap (1.17.1)
      msgpack (~> 1.2)
    bootstrap-sass (3.4.1)
      autoprefixer-rails (>= 5.2.1)
      sassc (>= 2.0.0)
    browser (2.5.3)
    builder (3.2.4)
    coderay (1.1.3)
    coffee-rails (5.0.0)
      coffee-script (>= 2.2.0)
      railties (>= 5.2.0)
    coffee-script (2.4.1)
      coffee-script-source
      execjs
    coffee-script-source (1.12.2)
    colored (1.2)
    combine_pdf (1.0.26)
      matrix
      ruby-rc4 (>= 0.1.5)
    concurrent-ruby (1.2.3)
    connection_pool (2.4.1)
    crass (1.0.6)
    date (3.3.4)
    debug (1.10.0)
      irb (~> 1.10)
      reline (>= 0.3.8)
    digest (3.1.1)
    docile (1.4.0)
    domain_name (0.6.20240107)
    dotenv (3.1.2)
    dotenv-rails (3.1.2)
      dotenv (= 3.1.2)
      railties (>= 6.1)
    drb (2.2.0)
      ruby2_keywords
    e2mmap (0.1.0)
    erubi (1.12.0)
    et-orbi (1.2.7)
      tzinfo
    ethon (0.16.0)
      ffi (>= 1.15.0)
    execjs (2.9.1)
    factory_bot (6.4.5)
      activesupport (>= 5.0.0)
    faraday (1.10.3)
      faraday-em_http (~> 1.0)
      faraday-em_synchrony (~> 1.0)
      faraday-excon (~> 1.1)
      faraday-httpclient (~> 1.0)
      faraday-multipart (~> 1.0)
      faraday-net_http (~> 1.0)
      faraday-net_http_persistent (~> 1.0)
      faraday-patron (~> 1.0)
      faraday-rack (~> 1.0)
      faraday-retry (~> 1.0)
      ruby2_keywords (>= 0.0.4)
    faraday-em_http (1.0.0)
    faraday-em_synchrony (1.0.0)
    faraday-excon (1.1.0)
    faraday-httpclient (1.0.1)
    faraday-multipart (1.0.4)
      multipart-post (~> 2)
    faraday-net_http (1.0.1)
    faraday-net_http_persistent (1.2.0)
    faraday-patron (1.0.0)
    faraday-rack (1.0.0)
    faraday-retry (1.0.3)
    ffi (1.17.0)
    ffi (1.17.0-aarch64-linux-gnu)
    ffi (1.17.0-arm-linux-gnu)
    ffi (1.17.0-arm64-darwin)
    ffi (1.17.0-x86-linux-gnu)
    ffi (1.17.0-x86_64-darwin)
    ffi (1.17.0-x86_64-linux-gnu)
    firefly_server (0.1.7)
      listen (~> 3.1)
    fugit (1.9.0)
      et-orbi (~> 1, >= 1.2.7)
      raabro (~> 1.4)
    geocodio (3.0.0)
      json
    globalid (1.2.1)
      activesupport (>= 6.1)
    http-cookie (1.0.5)
      domain_name (~> 0.5)
    i18n (1.14.1)
      concurrent-ruby (~> 1.0)
    importmap-rails (2.0.3)
      actionpack (>= 6.0.0)
      activesupport (>= 6.0.0)
      railties (>= 6.0.0)
    io-console (0.8.0)
    irb (1.15.1)
      pp (>= 0.6.0)
      rdoc (>= 4.0.0)
      reline (>= 0.4.2)
    jmespath (1.6.2)
    json (2.7.1)
    jwt (2.7.1)
    kgio (2.11.4)
    language_server-protocol (3.17.0.3)
    listen (3.8.0)
      rb-fsevent (~> 0.10, >= 0.10.3)
      rb-inotify (~> 0.9, >= 0.9.10)
    loofah (2.22.0)
      crass (~> 1.0.2)
      nokogiri (>= 1.12.0)
    mail (2.7.1)
      mini_mime (>= 0.1.1)
    marcel (1.0.2)
    matrix (0.4.2)
    method_source (1.0.0)
    mini_mime (1.1.5)
    mini_portile2 (2.8.7)
    minitest (5.21.2)
    minitest-ci (3.4.0)
      minitest (>= 5.0.6)
    msgpack (1.7.2)
    multi_json (1.15.0)
    multi_xml (0.6.0)
    multipart-post (2.3.0)
    mutex_m (0.2.0)
    net-imap (0.4.9.1)
      date
      net-protocol
    net-pop (0.1.2)
      net-protocol
    net-protocol (0.2.2)
      timeout
    net-sftp (4.0.0)
      net-ssh (>= 5.0.0, < 8.0.0)
    net-smtp (0.2.2)
      digest
      net-protocol
      timeout
    net-ssh (7.2.1)
    nio4r (2.7.0)
    nokogiri (1.16.7)
      mini_portile2 (~> 2.8.2)
      racc (~> 1.4)
    nokogiri (1.16.7-aarch64-linux)
      racc (~> 1.4)
    nokogiri (1.16.7-arm-linux)
      racc (~> 1.4)
    nokogiri (1.16.7-arm64-darwin)
      racc (~> 1.4)
    nokogiri (1.16.7-x86-linux)
      racc (~> 1.4)
    nokogiri (1.16.7-x86_64-darwin)
      racc (~> 1.4)
    nokogiri (1.16.7-x86_64-linux)
      racc (~> 1.4)
    oauth2 (1.4.11)
      faraday (>= 0.17.3, < 3.0)
      jwt (>= 1.0, < 3.0)
      multi_json (~> 1.3)
      multi_xml (~> 0.5)
      rack (>= 1.2, < 4)
    parallel (1.24.0)
    parser (3.3.1.0)
      ast (~> 2.4.1)
      racc
    pdf-core (0.9.0)
    pg (1.5.4)
    posthog-ruby (2.5.0)
      concurrent-ruby (~> 1)
    pp (0.6.2)
      prettyprint
    prawn (2.4.0)
      pdf-core (~> 0.9.0)
      ttfunk (~> 1.7)
    prettyprint (0.2.0)
    pry (0.14.2)
      coderay (~> 1.1)
      method_source (~> 1.0)
    pry-rails (0.3.9)
      pry (>= 0.10.4)
    psych (5.1.2)
      stringio
    puma (6.4.2)
      nio4r (~> 2.0)
    raabro (1.4.0)
    racc (1.8.1)
    rack (2.2.8)
    rack-protection (3.2.0)
      base64 (>= 0.1.0)
      rack (~> 2.2, >= 2.2.4)
    rack-test (2.1.0)
      rack (>= 1.3)
    rails (6.1.7.6)
      actioncable (= 6.1.7.6)
      actionmailbox (= 6.1.7.6)
      actionmailer (= 6.1.7.6)
      actionpack (= 6.1.7.6)
      actiontext (= 6.1.7.6)
      actionview (= 6.1.7.6)
      activejob (= 6.1.7.6)
      activemodel (= 6.1.7.6)
      activerecord (= 6.1.7.6)
      activestorage (= 6.1.7.6)
      activesupport (= 6.1.7.6)
      bundler (>= 1.15.0)
      railties (= 6.1.7.6)
      sprockets-rails (>= 2.0.0)
    rails-dom-testing (2.2.0)
      activesupport (>= 5.0.0)
      minitest
      nokogiri (>= 1.6)
    rails-html-sanitizer (1.6.0)
      loofah (~> 2.21)
      nokogiri (~> 1.14)
    railties (6.1.7.6)
      actionpack (= 6.1.7.6)
      activesupport (= 6.1.7.6)
      method_source
      rake (>= 12.2)
      thor (~> 1.0)
    rainbow (3.1.1)
    raindrops (0.20.1)
    rake (13.1.0)
    rb-fsevent (0.11.2)
    rb-inotify (0.10.1)
      ffi (~> 1.0)
    rdoc (6.12.0)
      psych (>= 4.0.0)
    record_tag_helper (1.0.1)
      actionview (>= 5)
    redis (4.5.1)
    regexp_parser (2.9.2)
    reline (0.6.0)
      io-console (~> 0.5)
    rexml (3.3.6)
      strscan
    rubocop (1.64.0)
      json (~> 2.3)
      language_server-protocol (>= 3.17.0)
      parallel (~> 1.10)
      parser (>= 3.3.0.2)
      rainbow (>= 2.2.2, < 4.0)
      regexp_parser (>= 1.8, < 3.0)
      rexml (>= 3.2.5, < 4.0)
      rubocop-ast (>= 1.31.1, < 2.0)
      ruby-progressbar (~> 1.7)
      unicode-display_width (>= 2.4.0, < 3.0)
    rubocop-ast (1.31.3)
      parser (>= 3.3.1.0)
    rubocop-factory_bot (2.25.1)
      rubocop (~> 1.41)
    rubocop-rails (2.25.0)
      activesupport (>= 4.2.0)
      rack (>= 1.1)
      rubocop (>= 1.33.0, < 2.0)
      rubocop-ast (>= 1.31.1, < 2.0)
    ruby-progressbar (1.13.0)
    ruby-rc4 (0.1.5)
    ruby-saml (1.17.0)
      nokogiri (>= 1.13.10)
      rexml
    ruby2_keywords (0.0.5)
    rufus-scheduler (3.9.1)
      fugit (~> 1.1, >= 1.1.6)
    sass-rails (6.0.0)
      sassc-rails (~> 2.1, >= 2.1.1)
    sassc (2.4.0)
      ffi (~> 1.9)
    sassc-rails (2.1.2)
      railties (>= 4.0.0)
      sassc (>= 2.0)
      sprockets (> 3.0)
      sprockets-rails
      tilt
    sentry-rails (5.17.3)
      railties (>= 5.0)
      sentry-ruby (~> 5.17.3)
    sentry-ruby (5.17.3)
      bigdecimal
      concurrent-ruby (~> 1.0, >= 1.0.2)
    sentry-sidekiq (5.17.3)
      sentry-ruby (~> 5.17.3)
      sidekiq (>= 3.0)
    sidekiq (5.2.7)
      connection_pool (~> 2.2, >= 2.2.2)
      rack (>= 1.5.0)
      rack-protection (>= 1.5.0)
      redis (>= 3.3.5, < 5)
    sidekiq-scheduler (3.0.1)
      e2mmap
      redis (>= 3, < 5)
      rufus-scheduler (~> 3.2)
      sidekiq (>= 3)
      thwait
      tilt (>= 1.4.0)
    simplecov (0.22.0)
      docile (~> 1.1)
      simplecov-html (~> 0.11)
      simplecov_json_formatter (~> 0.1)
    simplecov-html (0.12.3)
    simplecov_json_formatter (0.1.4)
    sprockets (4.2.1)
      concurrent-ruby (~> 1.0)
      rack (>= 2.2.4, < 4)
    sprockets-rails (3.4.2)
      actionpack (>= 5.2)
      activesupport (>= 5.2)
      sprockets (>= 3.0.0)
    stackprof (0.2.26)
    stimulus-rails (1.3.4)
      railties (>= 6.0.0)
    stringio (3.1.5)
    stripe (9.4.0)
    strscan (3.1.0)
    thor (1.3.0)
    thwait (0.2.0)
      e2mmap
    tilt (2.3.0)
    timeout (0.4.1)
    ttfunk (1.7.0)
    twilio-ruby (7.3.3)
      faraday (>= 0.9, < 3.0)
      jwt (>= 1.5, < 3.0)
      nokogiri (>= 1.6, < 2.0)
    typhoeus (1.4.1)
      ethon (>= 0.9.0)
    tzinfo (2.0.6)
      concurrent-ruby (~> 1.0)
    uglifier (4.2.0)
      execjs (>= 0.3.0, < 3)
    unicode-display_width (2.5.0)
    unicorn (6.1.0)
      kgio (~> 2.6)
      raindrops (~> 0.7)
    webrick (1.8.1)
    websocket-driver (0.7.6)
      websocket-extensions (>= 0.1.0)
    websocket-extensions (0.1.5)
    zeitwerk (2.6.12)
    zip_tricks (5.6.0)

PLATFORMS
  aarch64-linux
  arm-linux
  arm64-darwin
  ruby
  x86-linux
  x86_64-darwin
  x86_64-linux

DEPENDENCIES
  awesome_print
  aws-sdk-autoscaling
  aws-sdk-core (~> 3.48)
  aws-sdk-ec2
  aws-sdk-rds
  aws-sdk-s3
  bcrypt
  bigdecimal
  bootsnap
  bootstrap-sass (~> 3.4.1)
  browser (~> 2.5.3)
  coffee-rails
  colored
  combine_pdf (~> 1.0.23)
  debug (>= 1.0.0)
  dotenv-rails (~> 3.1.2)
  drb
  ethon
  factory_bot
  faraday (~> 1.0)
  firefly_server
  geocodio
  http-cookie
  importmap-rails (~> 2.0)
  listen
  mail (~> 2.7.1)
  method_source
  minitest-ci
  mutex_m
  net-imap
  net-pop
  net-sftp
  net-smtp (~> 0.2.0)
  net-ssh
  oauth2 (~> 1.4.3)
  pg
  posthog-ruby
  prawn (~> 2.4)
  pry
  pry-rails
  psych (~> 5.1.0)
  puma
  rails (~> 6.1)
  record_tag_helper (~> 1.0.1)
  redis (< 4.6)
  rexml
  rubocop
  rubocop-factory_bot
  rubocop-rails
  ruby-saml (~> 1.17.0)
  sass-rails (>= 5)
  sentry-rails
  sentry-ruby
  sentry-sidekiq
  sidekiq (~> 5.2.7)
  sidekiq-scheduler (~> 3.0.0)
  simplecov
  sprockets (~> 4)
  sprockets-rails
  stackprof
  stimulus-rails (~> 1.3)
  stripe (~> 9.4.0)
  thwait
  twilio-ruby (= 7.3.3)
  typhoeus
  uglifier
  unicorn (~> 6.1)
  webrick
  zip_tricks

RUBY VERSION
   ruby 3.3.0p0

BUNDLED WITH
   2.5.4

[robacarp]

Alrighty! Here's an even more minimal proof of concept, without Rails, ActiveSupport, etc.

Running it is trivial, and the instructions are in the readme.

I was able to upgrade Puma to 6.6.0 (latest) and still trigger the bug.
I also tried out latest public stable ruby 3.4.1 and no change.

However switching to unicorn resolves the issue. I briefly tried thin but it failed to install on the first try and I didn't want to get distracted.

[arenclissold]

Hey I ran into this today. Thanks for the discussion above.
If it helps I noticed that when running 1 worker the hanging issue where you cannot continue/quit no longer occurs, although the issue where you attach to the puma worker process when a debugger hasn't been hit still does

[Ikariusrb]

I found that adding Phlex to an absolute minimal rails codebase triggers this, breaking ruby/debug completely. Latest ruby, rails, ruby/debug, and puma. yippee-fun/phlex#958