From Discord: Getting Input from Pipe Operator

[effect-bot]

Summary

In the discussion, participants are exploring how to handle input from a pipe operator in a TypeScript CLI application using the Effect library. Key points include:

1. Standard Input (stdin): Typically accessed in Node.js via process.stdin. The library possibly has utilities for handling input but lacks direct support for streaming stdin continuously.

2. Arguments (Args) and stdin: The traditional way of parsing command line options doesn't directly support piping input as arguments.

3. Proposal for Enhancement:

   • Introduce an Arg combinator that can handle stdin as if it were an argument.
   • Consider using a symbol like - to indicate reading from stdin, mimicking conventions from other CLI tools.
   • Explore using Streams in conjunction with Args, allowing a stream from stdin to be treated as part of the argument parsing process.

4. Challenges and Considerations:

   • Handling streaming input could potentially cause a program to hang if it waits indefinitely for input.
   • Designing a system where the application decides how to process the potentially streaming data.

5. Implementation Ideas:

   • Introduce a method to declare Args that can incorporate stdin, possibly resulting in a Stream<string>.
   • Such features could be beneficial for processing command-line arguments intended to handle multiple inputs or streaming content.

The conversation involves contributions from several community members, considering both current limitations and potential enhancements to better handle piped input in CLI applications.

Discord thread

https://discord.com/channels/795981131316985866/1234796735730286654

[joepjoosten]

I'm using this code to switch from reading an argument from the input to stdin:

import { Args } from '@effect/cli';
import { Array, flow, Function, pipe, Schema, Stream, String } from 'effect';

const stdinStream = () =>
  Stream.fromAsyncIterable(process.stdin as AsyncIterable<Uint8Array>, () => new Error('Failed to read from stdin'));
const IFS = process.env.IFS || '\n';
const splitByIFS = flow(String.split(IFS), Array.map(String.trim), Array.filter(String.isNonEmpty));

const stdinAsString = Stream.mapEffect((chunk: Uint8Array) =>
  Schema.decode(
    Schema.Uint8ArrayFromSelf.pipe(
      Schema.transform(Schema.String, {
        strict: true,
        encode: (input) => new TextEncoder().encode(input),
        decode: (input) => input.toString(),
      }),
    ),
  )(chunk),
);

export const argsFromStdin = ({ readFromStdinDash = '-', useIFS = true, decode = stdinAsString }) =>
  Args.map((input: string) =>
    input === readFromStdinDash
      ? pipe(stdinStream(), decode, Stream.mapConcat(useIFS ? splitByIFS : Function.identity))
      : Stream.make(input),
  );

Then you can use it like this:

const strArg = Args.text({ name: 'str' }).pipe(
  argsFromStdin({}),
);

// Define CLI command
const command = Command.make(
  'read-stdin',
  {
    strArg,
  },
  ({ strArg }) =>
    pipe(
      domain,
      Stream.tap((strArg) => Console.log(`test: ${strArg}`)),
      Stream.runDrain,
    ),
);

// Configure and initialize the CLI application
const cli = Command.run(command, {
  name: 'Echo CLI',
  version: 'v0.0.1',
});

cli(process.argv).pipe(Effect.provide(NodeContext.layer), NodeRuntime.runMain);

call it like this (using the dash as a indicator to read from stdin):

echo "test" | tsx command.ts -