Can I auto encode string arguments to shell commands to simply gh operations?

[bhouston]

I feel that I could auto encode end of line characters in string arguments.

I think the approaches are different on windows than on macOS and Linux.

But this would simplify the GitHub integration significantly.

[bhouston]

Proposal for Auto-Encoding String Arguments in Shell Commands

Current Situation

Currently, when working with GitHub CLI commands that require multiline content (like issue or PR descriptions), the tool writes content to temporary files on disk and references them with --body-file. This approach works but is awkward and requires cleanup of temporary files.

The core issue is properly escaping string arguments with line endings across different operating systems (Windows, macOS, Linux) when passing them directly to shell commands.

Research on Available Solutions

After researching various approaches, I've identified several potential solutions:

1. Use existing shell escaping libraries

There are several npm packages that can help with shell escaping:

shell-escape (286 dependent projects)

• Escapes and stringifies an array of arguments for shell execution
• Simple API: shellescape(['echo', 'hello world', 'how\'s it going?'])
• Last updated 10 years ago
• May not handle Windows shell escaping properly

cross-spawn (10,000+ dependent projects)

• Cross-platform solution for Node.js's child_process.spawn()
• Handles Windows/POSIX differences automatically
• More actively maintained
• Focuses on argument handling rather than direct string escaping

escape-string-regexp

• Not directly applicable as it's for RegExp escaping, not shell commands

2. Platform-specific approach

Shell escaping rules differ significantly between platforms:

• POSIX (Linux/macOS): Uses single quotes to escape strings (doubling single quotes within the string)
• Windows CMD: Uses carets (^) and double quotes differently
• PowerShell: Has its own escaping rules with backticks

3. Base64 encoding approach

Instead of escaping, we could encode multiline content:

1. Base64 encode the string content
2. Pass the encoded string to the shell command
3. Have the shell command decode it before use

Example with GitHub CLI:

const encodedContent = Buffer.from(content).toString('base64');
await shellExecute({
  command: `echo "${encodedContent}" | base64 -d | gh issue create --body-stdin`,
  description: 'Creating issue with encoded content'
});

This approach works consistently across platforms as base64 is standard.

Proposed Solution

I recommend implementing a combination of approaches:

1. For simple commands: Use the cross-spawn package to handle argument escaping properly across platforms.

2. For complex multiline content: Implement a pipe-based approach using base64 encoding:

   • Create a utility function executeWithEncodedInput(command, input, options)
   • The function encodes the input and pipes it to the command
   • Works with any command that accepts stdin input

3. Add platform detection: Detect the current platform and use the appropriate escaping strategy:

   • For Windows: Use PowerShell's encoding capabilities when appropriate
   • For POSIX: Use base64 encoding through pipes

Implementation Example

import { spawn } from 'child_process';
import crossSpawn from 'cross-spawn';

// For simple commands with arguments that need escaping
function executeCommand(command: string, args: string[], options = {}) {
  return crossSpawn(command, args, { ...options, shell: true });
}

// For commands with complex multiline input
async function executeWithEncodedInput(command: string, input: string, options = {}) {
  const isWindows = process.platform === 'win32';
  
  if (isWindows) {
    // Windows-specific approach using PowerShell
    const encodedCommand = Buffer.from(input).toString('base64');
    return crossSpawn('powershell', [
      '-Command',
      `[System.Text.Encoding]::UTF8.GetString([System.Convert]::FromBase64String('${encodedCommand}')) | ${command}`
    ], options);
  } else {
    // POSIX approach (Linux/macOS)
    const encodedInput = Buffer.from(input).toString('base64');
    const child = crossSpawn('bash', [
      '-c',
      `echo "${encodedInput}" | base64 -d | ${command}`
    ], options);
    
    return child;
  }
}

Benefits

1. Cross-platform compatibility: Works consistently on Windows, macOS, and Linux
2. No temporary files needed: Eliminates the need to create and clean up temporary files
3. Handles all special characters: Base64 encoding preserves all characters, including newlines, quotes, etc.
4. Simpler API: Provides a cleaner interface for tools that use shell commands

Next Steps

1. Implement the proposed utility functions
2. Add tests for various edge cases (special characters, very long strings, etc.)
3. Update the GitHub integration to use the new approach
4. Document the new functionality for users

Would you like me to proceed with implementing this solution?

[bhouston]

Thank you for the research. I think that the best way to make this would and to have the tool parameters understandable intuitively by the LLM would be to add a parameter called stdinContent (a string) to the shellStart and shellExecute functions and describe it as content that is pipelined into the shell command upon start, or something to that extent. We can then use crossSpawn to start it in the cross platform way that you demonstrate above.

[bhouston]

Updating the GitHub integration is really just changing the prompt used for githubMode so that it says use stdinContent for the body of GitHub issues, pull requests and comments rather than temporary files.

[github-actions]

🎉 This issue has been resolved in version mycoder-agent-v1.5.0 🎉

The release is available on:

• pnpm package (@latest dist-tag)
• GitHub release

Your semantic-release bot 📦🚀

[github-actions]

🎉 This issue has been resolved in version mycoder-v1.5.0 🎉

The release is available on:

• pnpm package (@latest dist-tag)
• GitHub release

Your semantic-release bot 📦🚀