[BUG] ellipsis rendering fails for non-unicode encoding in tables

[dlax]

• I've checked docs and closed issues for possible solutions.
• I can't find my issue in the FAQ.

Describe the bug

When using a non-unicode encoding, e.g. "ascii", rendering of ellipsis in truncated table cells fails:

# file: t.py
from rich.console import Console
from rich.table import Table

console = Console(width=10)
print(f"{console.encoding=}")

t = Table("column")
t.add_row("x" * 10)
console.print(t)

$ env PYTHONIOENCODING=ascii python t.py
console.encoding='ascii'
Traceback (most recent call last):
  File "/tmp/t.py", line 9, in <module>
    console.print(t)
    ~~~~~~~~~~~~~^^^
  File ".../rich/console.py", line 1692, in print
    with self:
         ^^^^
  File ".../rich/console.py", line 866, in __exit__
    self._exit_buffer()
    ~~~~~~~~~~~~~~~~~^^
  File ".../rich/console.py", line 824, in _exit_buffer
    self._check_buffer()
    ~~~~~~~~~~~~~~~~~~^^
  File ".../rich/console.py", line 2033, in _check_buffer
    self._write_buffer()
    ~~~~~~~~~~~~~~~~~~^^
  File "...rich/console.py", line 2102, in _write_buffer
    self.file.write(text)
    ~~~~~~~~~~~~~~~^^^^^^
UnicodeEncodeError: 'ascii' codec can't encode character '\u2026' in position 64: ordinal not in range(128)
*** You may need to add PYTHONIOENCODING=utf-8 to your environment ***

As a workaround, one can force the "overflow" method per-column:

t = Table()
t.add_column("column", overflow="fold")
t.add_row("x" * 10)

but this is a bit cumbersome and in fact prevents usage of "ellipsis" overflow method.

As the rest of the table, borders in particular, is aware of the encoding and have its rendering adjusted, I wonder if text truncation in Text.truncate() should have the same fallback and use an ASCII character (to be determined)?

Platform

Click to expand

What platform (Win/Linux/Mac) are you running on? What terminal software are you using?
Alacritty, on Linux.

I may ask you to copy and paste the output of the following commands. It may save some time if you do it now.

If you're using Rich in a terminal:

$ python -m rich.diagnose
+---------------------- <class 'rich.console.Console'> -----------------------+
| A high level console interface.                                             |
|                                                                             |
| +-------------------------------------------------------------------------+ |
| | <console width=79 ColorSystem.TRUECOLOR>                                | |
| +-------------------------------------------------------------------------+ |
|                                                                             |
|     color_system = 'truecolor'                                              |
|         encoding = 'ascii'                                                  |
|             file = <_io.TextIOWrapper name='<stdout>' mode='w'              |
|                    encoding='ascii'>                                        |
|           height = 49                                                       |
|    is_alt_screen = False                                                    |
| is_dumb_terminal = False                                                    |
|   is_interactive = True                                                     |
|       is_jupyter = False                                                    |
|      is_terminal = True                                                     |
|   legacy_windows = False                                                    |
|         no_color = False                                                    |
|          options = ConsoleOptions(                                          |
|                        size=ConsoleDimensions(width=79, height=49),         |
|                        legacy_windows=False,                                |
|                        min_width=1,                                         |
|                        max_width=79,                                        |
|                        is_terminal=True,                                    |
|                        encoding='ascii',                                    |
|                        max_height=49,                                       |
|                        justify=None,                                        |
|                        overflow=None,                                       |
|                        no_wrap=False,                                       |
|                        highlight=None,                                      |
|                        markup=None,                                         |
|                        height=None                                          |
|                    )                                                        |
|            quiet = False                                                    |
|           record = False                                                    |
|         safe_box = True                                                     |
|             size = ConsoleDimensions(width=79, height=49)                   |
|        soft_wrap = False                                                    |
|           stderr = False                                                    |
|            style = None                                                     |
|         tab_size = 8                                                        |
|            width = 79                                                       |
+-----------------------------------------------------------------------------+
+--- <class 'rich._windows.WindowsConsoleFeatures'> ----+
| Windows features available.                           |
|                                                       |
| +---------------------------------------------------+ |
| | WindowsConsoleFeatures(vt=False, truecolor=False) | |
| +---------------------------------------------------+ |
|                                                       |
| truecolor = False                                     |
|        vt = False                                     |
+-------------------------------------------------------+
+------ Environment Variables -------+
| {                                  |
|     'CLICOLOR': None,              |
|     'COLORTERM': 'truecolor',      |
|     'COLUMNS': None,               |
|     'JPY_PARENT_PID': None,        |
|     'JUPYTER_COLUMNS': None,       |
|     'JUPYTER_LINES': None,         |
|     'LINES': None,                 |
|     'NO_COLOR': None,              |
|     'TERM_PROGRAM': None,          |
|     'TERM': 'alacritty',           |
|     'TTY_COMPATIBLE': None,        |
|     'VSCODE_VERBOSE_LOGGING': None |
| }                                  |
+------------------------------------+
platform="Linux"

$ pip freeze | grep rich
-e git+https://github.com/willmcgugan/rich@c8f234f136b3ec60da6a96364705490b7ee9cf39#egg=rich

[github-actions]

Thank you for your issue. Give us a little time to review it.

PS. You might want to check the FAQ if you haven't done so already.

This is an automated reply, generated by FAQtory

[willmcgugan]

There isn't a single character that would make sense that I know of. The only options would be to use three dots, or ignore the ellipsis and overflow.

I don't think it is practical to guard against all ascii encoding issues. There is so much that Rich does, that doesn't have some kind of ascii fallback.

It may also be redundant. New Pythons uses utf-8 by default. Why would you ever want to encode something as ascii?

[dlax]

There isn't a single character that would make sense that I know of.

In this specific case, for tables, I think using no character, as would result Text.truncate(..., overflow=None), is acceptable. This changes does the trick:

diff --git a/rich/containers.py b/rich/containers.py
index 901ff8ba..63eb6473 100644
--- a/rich/containers.py
+++ b/rich/containers.py
@@ -113,7 +113,7 @@ class Lines:
         console: "Console",
         width: int,
         justify: "JustifyMethod" = "left",
-        overflow: "OverflowMethod" = "fold",
+        overflow: Optional["OverflowMethod"] = "fold",
     ) -> None:
         """Justify and overflow text to a given width.

@@ -126,6 +126,9 @@ class Lines:
         """
         from .text import Text

+        if overflow == "ellipsis" and console.options.ascii_only:
+            overflow = None
+
         if justify == "left":
             for line in self._lines:
                 line.truncate(width, overflow=overflow, pad=True)

I don't think it is practical to guard against all ascii encoding issues. There is so much that Rich does, that doesn't have some kind of ascii fallback.

I understand that and it's totally acceptable. But on the other hand, tables already have a guard and a fallback, but it's not complete; so my question is really about whether it should be extended or not. I'm happy to contribute a change if you think it's worth. Let me know either way.

Why would you ever want to encode something as ascii?

In my case, it happened outside my control as someone was using a program I maintain in an environment set up with ascii encoding; asking them to alter that environment for utf-8 is certainly acceptable (and in fact, we did).

[willmcgugan]

That change wouldn't catch it everywhere. Text.truncate could be invoked from other places, and that method doesn't have access to the console object. I'm not keen on patching it just for tables.

I'm going to close this for now, because I can't think of a generalized solution. I would like to address it. Feel free to re-open if you have any ideas on that front.

[github-actions]

I hope I helped!

Consider sponsoring my work on Rich. I give tech support for free, in addition to maintaining Rich and Textual.

If you like using Rich, you might also enjoy Textual.

Will McGugan