Menu

#105 Win32 Unicode (UTF-8) support

open
nobody
None
5
2024-11-30
2015-03-21
MuldeR
No

On the Windows platform, SoX currently supports file names in the user's current ANSI Codepage only - whatever Codepage that happens to be - but it fails horribly with any "foreign" characters in the file name. This patch resolves the problem. It's essential for Windows users.

Put simply, command-line parameters are retrieved using the Wide-String (UTF-16) Win32 functions and then converted to UTF-8 internally, so all "char"-Strings inside SoX will be in proper UTF-8 encoding in Windows too (just like in Linux, where we get that "for free"). Also, whenever any System functions, like fopen(), stat() or unlink() are used on Windows, we convert back to Wide-String and call the Wide-String version. This is required, because in Windows the "char"-String versions of these functions do NOT support Unicode/UTF-8 at all!

This all is handled transparently in the "unicode_support.c" file. Changes to original SoX code are restricted to a minimum. A similar patch was committed into Opus Tools, by the way.

1 Attachments

Related

Patches: #105

Discussion

  • J. P.

    J. P. - 2015-05-13

    Hello,

    I am attempting to use your binary to SoX and I am having a little difficulty.

    I assume it is a direct replace for the official Sox binary release for Windows. However, when I replace it, yours was unable to see the helper libraries. So, with the original I could process FLAC files (just had issues with Unicode file names), and with yours, I get an error that it is unable to process FLAC files.

    Please advise.

     
    • MuldeR

      MuldeR - 2015-05-13

      Hello.

      My build is kind of a "minimal" build with no extra lib's enabled. This
      binary is a byproduct of my LameXP software, where those extra lib's are
      simply not needed. Sorry.

      If you need those extra lib's, you will have to build yourself with my
      UTF-8 patch and the extra lib's enabled. Things would of course be
      much better, if they simply adopted my UTF-8 patch (or another solution
      for proper Unicode support).

      Regards,
      MuldeR

      On 13.05.2015 20:44, J. P. wrote:

      Hello,

      I am attempting to use your binary to SoX and I am having a little
      difficulty.

      I assume it is a direct replace for the official Sox binary release
      for Windows. However, when I replace it, yours was unable to see the
      helper libraries. So, with the original I could process FLAC files
      (just had issues with Unicode file names), and with yours, I get an
      error that it is unable to process FLAC files.

      Please advise.


      [patches:#105] http://sourceforge.net/p/sox/patches/105 Win32
      Unicode (UTF-8) support

      Status: open
      Group:
      Created: Sat Mar 21, 2015 01:56 PM UTC by MuldeR
      Last Updated: Sat Mar 21, 2015 01:56 PM UTC
      Owner: nobody

      On the Windows platform, SoX currently supports file names in the
      user's current ANSI Codepage only - whatever Codepage that happens to
      be - but it fails horribly with any "foreign" characters in the file
      name. This patch resolves the problem. It's essential for Windows users.

      Put simply, command-line parameters are retrieved using the
      Wide-String (UTF-16) Win32 functions and then converted to UTF-8
      internally, so all "char"-Strings inside SoX will be in proper UTF-8
      encoding in Windows too (just like in Linux, where we get that "for
      free"). Also, whenever any System functions, like fopen(), stat() or
      unlink() are used on Windows, we convert back to Wide-String and call
      the Wide-String version. This is required, because in Windows the
      "char"-String versions of these functions do NOT support Unicode/UTF-8
      at all!

      This all is handled transparently in the "unicode_support.c" file.
      Changes to original SoX code are restricted to a minimum. A similar
      patch was committed into Opus Tools, by the way.


      Sent from sourceforge.net because you indicated interest in
      https://sourceforge.net/p/sox/patches/105/
      https://sourceforge.net/p/sox/patches/105

      To unsubscribe from further messages, please visit
      https://sourceforge.net/auth/subscriptions/
      https://sourceforge.net/auth/subscriptions

      --
      https://www.youtube.com/v/VF9UMona74w?autoplay=1

       

      Related

      Patches: #105

  • J. P.

    J. P. - 2015-05-14

    Thanks

     
  • ginseng2

    ginseng2 - 2016-06-27

    Hi MuldeR,

    Thanks for the patch, yet sox pipes doesn't work anymore! Any chance to fix that? ie.

    sox "|sox -m file1.mp3 file2.mp3 -p" "|sox -m file1.mp3 file6.mp3 file7.mp3 -p" result.mp3

     

Log in to post a comment.

Want the latest updates on software, tech news, and AI?
Get latest updates about software, tech news, and AI from SourceForge directly in your inbox once a month.