fix: SQL Context 에러 수정 및 Streamlit UI 파싱 수정, 웹페이지 클리닝 #91

ParkGyeongTae · 2025-05-02T04:11:34Z

#️⃣ Issue Number

쿼리 실행시 간헐적으로 에러 발생 : AttributeError: 'AIMessage' object has no attribute 'encode' #87

📝 요약(Summary)

SQL Context 에러 수정 및 Streamlit UI 파싱 수정, 웹페이지 클리닝

💬 To Reviewers (선택)

없음

PR Checklist

lang2sql --datahub_server http://34.125.222.80:8080 run-streamlit

reference) How to Code Review

따봉(👍): 리뷰어가 리뷰이의 코드에서 칭찬의 의견을 남기고 싶을 때 사용합니다.
느낌표(❗): 리뷰어가 리뷰이에게 필수적으로 코드 수정을 요청할 때 사용합니다.
물음표 (❓): 리뷰어가 리뷰이에게 의견을 물어보고 싶을 때 사용합니다.
알약 (💊): 리뷰어가 리뷰이의 코드에서 개선된 방법을 제안하지만 그것의 반영이 필수까지는 아닐 때 사용합니다.

ParkGyeongTae · 2025-05-02T04:13:40Z

res["generated_query"]의 리턴값이 문자열일때도 있고, AIMessage일때도 있는 현상 해결함.
아래는 에러 내용임.

2025-05-02 11:46:52 [INFO] Successfully connected to ClickHouse.
2025-05-02 11:46:52 [INFO] Loading faiss with AVX2 support.
2025-05-02 11:46:52 [INFO] Successfully loaded faiss with AVX2 support.
2025-05-02 11:46:52 [INFO] Failed to load GPU Faiss: name 'GpuIndexIVFFlat' is not defined. Will not load constructor refs for GPU indexes.
2025-05-02 11:46:54 [INFO] HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
2025-05-02 11:46:55 [INFO] HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK"
2025-05-02 11:46:58 [INFO] HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK"
content='<SQL>\n```sql\n    SELECT COUNT(DISTINCT entity_id)\n    FROM client_stream_ordered_service\n    WHERE activity_ts >= date_sub(now(), interval 3 month)\n    AND (JSON_CONTAINS(feature_json, \'"service1"\')\n         OR JSON_CONTAINS(feature_json, \'"service2"\')\n         OR JSON_CONTAINS(feature_json, \'"service3"\'))\n```\n\n<해석>\n```plaintext\n    이 쿼리는 client_stream_ordered_service 테이블에서 최근 3개월 동안 service1, service2, service3\n    중 하나의 서비스를 주문한 고객의 고유한 수를 계산합니다.\n    activity_ts 컬럼을 이용해 최근 3개월로 기간을 제한하고, feature_json 컬럼에서 지정된 서비스를\n    포함하고 있는지 확인하여 필터링합니다. entity_id 컬럼을 사용하여 고유 고객 수를 집계합니다.\n```' additional_kwargs={'refusal': None} response_metadata={'token_usage': {'completion_tokens': 191, 'prompt_tokens': 935, 'total_tokens': 1126, 'completion_tokens_details': {'accepted_prediction_tokens': 0, 'audio_tokens': 0, 'reasoning_tokens': 0, 'rejected_prediction_tokens': 0}, 'prompt_tokens_details': {'audio_tokens': 0, 'cached_tokens': 0}}, 'model_name': 'gpt-4o-2024-08-06', 'system_fingerprint': 'fp_55d88aaf2f', 'finish_reason': 'stop', 'logprobs': None} id='run-dc9b75e6-0230-4ab0-a7d3-509844036057-0' usage_metadata={'input_tokens': 935, 'output_tokens': 191, 'total_tokens': 1126, 'input_token_details': {'audio': 0, 'cache_read': 0}, 'output_token_details': {'audio': 0, 'reasoning': 0}}
2025-05-02 11:46:59 [ERROR] An error occurred while executing SQL: 'AIMessage' object has no attribute 'encode'
Traceback (most recent call last):
  File "/Users/pgt0409/Desktop/git/Lang2SQL/llm_utils/connect_db.py", line 95, in run_sql
    result = self.client.execute(sql, with_column_types=True)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/pgt0409/opt/anaconda3/envs/lang2sql/lib/python3.12/site-packages/clickhouse_driver/client.py", line 382, in execute
    rv = self.process_ordinary_query(
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/pgt0409/opt/anaconda3/envs/lang2sql/lib/python3.12/site-packages/clickhouse_driver/client.py", line 577, in process_ordinary_query
    self.connection.send_query(query, query_id=query_id, params=params)
  File "/Users/pgt0409/opt/anaconda3/envs/lang2sql/lib/python3.12/site-packages/clickhouse_driver/connection.py", line 723, in send_query
    write_binary_str(query, self.fout)
  File "/Users/pgt0409/opt/anaconda3/envs/lang2sql/lib/python3.12/site-packages/clickhouse_driver/writer.py", line 15, in write_binary_str
    text = text.encode('utf-8')
           ^^^^^^^^^^^
  File "/Users/pgt0409/opt/anaconda3/envs/lang2sql/lib/python3.12/site-packages/pydantic/main.py", line 891, in __getattr__
    raise AttributeError(f'{type(self).__name__!r} object has no attribute {item!r}')
AttributeError: 'AIMessage' object has no attribute 'encode'
2025-05-02 11:46:59.458 Uncaught app execution
Traceback (most recent call last):
  File "/Users/pgt0409/opt/anaconda3/envs/lang2sql/lib/python3.12/site-packages/streamlit/runtime/scriptrunner/exec_code.py", line 88, in exec_func_with_error_handling
    result = func()
             ^^^^^^
  File "/Users/pgt0409/opt/anaconda3/envs/lang2sql/lib/python3.12/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 579, in code_to_exec
    exec(code, module.__dict__)
  File "/Users/pgt0409/Desktop/git/Lang2SQL/interface/streamlit_app.py", line 75, in <module>
    st.navigation(pages).run()
  File "/Users/pgt0409/opt/anaconda3/envs/lang2sql/lib/python3.12/site-packages/streamlit/navigation/page.py", line 303, in run
    exec(code, module.__dict__)
  File "/Users/pgt0409/Desktop/git/Lang2SQL/interface/lang2sql.py", line 187, in <module>
    display_result(res=result, database=db)
  File "/Users/pgt0409/Desktop/git/Lang2SQL/interface/lang2sql.py", line 120, in display_result
    df = database.run_sql(sql)
         ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/pgt0409/Desktop/git/Lang2SQL/llm_utils/connect_db.py", line 95, in run_sql
    result = self.client.execute(sql, with_column_types=True)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/pgt0409/opt/anaconda3/envs/lang2sql/lib/python3.12/site-packages/clickhouse_driver/client.py", line 382, in execute
    rv = self.process_ordinary_query(
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/pgt0409/opt/anaconda3/envs/lang2sql/lib/python3.12/site-packages/clickhouse_driver/client.py", line 577, in process_ordinary_query
    self.connection.send_query(query, query_id=query_id, params=params)
  File "/Users/pgt0409/opt/anaconda3/envs/lang2sql/lib/python3.12/site-packages/clickhouse_driver/connection.py", line 723, in send_query
    write_binary_str(query, self.fout)
  File "/Users/pgt0409/opt/anaconda3/envs/lang2sql/lib/python3.12/site-packages/clickhouse_driver/writer.py", line 15, in write_binary_str
    text = text.encode('utf-8')
           ^^^^^^^^^^^
  File "/Users/pgt0409/opt/anaconda3/envs/lang2sql/lib/python3.12/site-packages/pydantic/main.py", line 891, in __getattr__
    raise AttributeError(f'{type(self).__name__!r} object has no attribute {item!r}')
AttributeError: 'AIMessage' object has no attribute 'encode'

ParkGyeongTae · 2025-05-02T04:15:22Z

쿼리를 실행하는 부분에 간헐적으로 한글이 같이 껴있는 현상 수정.
아래는 에러 내용.

2025-05-02 11:56:00 [INFO] Successfully connected to ClickHouse.
2025-05-02 11:56:01 [INFO] HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
2025-05-02 11:56:02 [INFO] HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK"
2025-05-02 11:56:05 [INFO] HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK"
2025-05-02 11:56:05 [ERROR] An error occurred while executing SQL: Code: 62.
DB::Exception: Unrecognized token: Syntax error: failed at position 343 (해) (line 10, col 2): 해석>

plaintext
    이 쿼리는 client_stream_ordered_service 테이블에서 최근 1개월 동안의 데이터를 기준으로
    각 서비스별('service1', 'service2', 'service3') ... . Stack trace:

0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000f48e57b
1. DB::Exception::createDeprecated(String const&, int, bool) @ 0x000000000f5528cd
2. DB::parseQueryAndMovePosition(DB::IParser&, char const*&, char const*, String const&, bool, unsigned long, unsigned long, unsigned long) @ 0x0000000015730f72
3. DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*, std::shared_ptr<DB::IAST>&) @ 0x00000000138fc873
4. DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x00000000138fbba4
5. DB::TCPHandler::runImpl() @ 0x0000000014c2aaec
6. DB::TCPHandler::run() @ 0x0000000014c48cd9
7. Poco::Net::TCPServerConnection::start() @ 0x00000000183727c7
8. Poco::Net::TCPServerDispatcher::run() @ 0x0000000018372c19
9. Poco::PooledThread::run() @ 0x000000001833ef5b
10. Poco::ThreadImpl::runnableEntry(void*) @ 0x000000001833d43d
11. ? @ 0x00007f1fb820eac3
12. ? @ 0x00007f1fb829fa04
Traceback (most recent call last):
  File "/Users/pgt0409/Desktop/git/Lang2SQL/llm_utils/connect_db.py", line 95, in run_sql
    result = self.client.execute(sql, with_column_types=True)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/pgt0409/opt/anaconda3/envs/lang2sql/lib/python3.12/site-packages/clickhouse_driver/client.py", line 382, in execute
    rv = self.process_ordinary_query(
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/pgt0409/opt/anaconda3/envs/lang2sql/lib/python3.12/site-packages/clickhouse_driver/client.py", line 580, in process_ordinary_query
    return self.receive_result(with_column_types=with_column_types,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/pgt0409/opt/anaconda3/envs/lang2sql/lib/python3.12/site-packages/clickhouse_driver/client.py", line 212, in receive_result
    return result.get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/Users/pgt0409/opt/anaconda3/envs/lang2sql/lib/python3.12/site-packages/clickhouse_driver/result.py", line 50, in get_result
    for packet in self.packet_generator:
                  ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/pgt0409/opt/anaconda3/envs/lang2sql/lib/python3.12/site-packages/clickhouse_driver/client.py", line 228, in packet_generator
    packet = self.receive_packet()
             ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/pgt0409/opt/anaconda3/envs/lang2sql/lib/python3.12/site-packages/clickhouse_driver/client.py", line 245, in receive_packet
    raise packet.exception
clickhouse_driver.errors.ServerException: Code: 62.
DB::Exception: Unrecognized token: Syntax error: failed at position 343 (해) (line 10, col 2): 해석>

plaintext
    이 쿼리는 client_stream_ordered_service 테이블에서 최근 1개월 동안의 데이터를 기준으로
    각 서비스별('service1', 'service2', 'service3') ... . Stack trace:

0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000f48e57b
1. DB::Exception::createDeprecated(String const&, int, bool) @ 0x000000000f5528cd
2. DB::parseQueryAndMovePosition(DB::IParser&, char const*&, char const*, String const&, bool, unsigned long, unsigned long, unsigned long) @ 0x0000000015730f72
3. DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*, std::shared_ptr<DB::IAST>&) @ 0x00000000138fc873
4. DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x00000000138fbba4
5. DB::TCPHandler::runImpl() @ 0x0000000014c2aaec
6. DB::TCPHandler::run() @ 0x0000000014c48cd9
7. Poco::Net::TCPServerConnection::start() @ 0x00000000183727c7
8. Poco::Net::TCPServerDispatcher::run() @ 0x0000000018372c19
9. Poco::PooledThread::run() @ 0x000000001833ef5b
10. Poco::ThreadImpl::runnableEntry(void*) @ 0x000000001833d43d
11. ? @ 0x00007f1fb820eac3
12. ? @ 0x00007f1fb829fa04

2025-05-02 11:56:05.752 Uncaught app execution
Traceback (most recent call last):
  File "/Users/pgt0409/opt/anaconda3/envs/lang2sql/lib/python3.12/site-packages/streamlit/runtime/scriptrunner/exec_code.py", line 88, in exec_func_with_error_handling
    result = func()
             ^^^^^^
  File "/Users/pgt0409/opt/anaconda3/envs/lang2sql/lib/python3.12/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 579, in code_to_exec
    exec(code, module.__dict__)
  File "/Users/pgt0409/Desktop/git/Lang2SQL/interface/streamlit_app.py", line 75, in <module>
    st.navigation(pages).run()
  File "/Users/pgt0409/opt/anaconda3/envs/lang2sql/lib/python3.12/site-packages/streamlit/navigation/page.py", line 303, in run
    exec(code, module.__dict__)
  File "/Users/pgt0409/Desktop/git/Lang2SQL/interface/lang2sql.py", line 188, in <module>
    display_result(res=result, database=db)
  File "/Users/pgt0409/Desktop/git/Lang2SQL/interface/lang2sql.py", line 121, in display_result
    df = database.run_sql(sql)
         ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/pgt0409/Desktop/git/Lang2SQL/llm_utils/connect_db.py", line 95, in run_sql
    result = self.client.execute(sql, with_column_types=True)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/pgt0409/opt/anaconda3/envs/lang2sql/lib/python3.12/site-packages/clickhouse_driver/client.py", line 382, in execute
    rv = self.process_ordinary_query(
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/pgt0409/opt/anaconda3/envs/lang2sql/lib/python3.12/site-packages/clickhouse_driver/client.py", line 580, in process_ordinary_query
    return self.receive_result(with_column_types=with_column_types,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/pgt0409/opt/anaconda3/envs/lang2sql/lib/python3.12/site-packages/clickhouse_driver/client.py", line 212, in receive_result
    return result.get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/Users/pgt0409/opt/anaconda3/envs/lang2sql/lib/python3.12/site-packages/clickhouse_driver/result.py", line 50, in get_result
    for packet in self.packet_generator:
                  ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/pgt0409/opt/anaconda3/envs/lang2sql/lib/python3.12/site-packages/clickhouse_driver/client.py", line 228, in packet_generator
    packet = self.receive_packet()
             ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/pgt0409/opt/anaconda3/envs/lang2sql/lib/python3.12/site-packages/clickhouse_driver/client.py", line 245, in receive_packet
    raise packet.exception
clickhouse_driver.errors.ServerException: Code: 62.
DB::Exception: Unrecognized token: Syntax error: failed at position 343 (해) (line 10, col 2): 해석>

plaintext
    이 쿼리는 client_stream_ordered_service 테이블에서 최근 1개월 동안의 데이터를 기준으로
    각 서비스별('service1', 'service2', 'service3') ... . Stack trace:

0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000f48e57b
1. DB::Exception::createDeprecated(String const&, int, bool) @ 0x000000000f5528cd
2. DB::parseQueryAndMovePosition(DB::IParser&, char const*&, char const*, String const&, bool, unsigned long, unsigned long, unsigned long) @ 0x0000000015730f72
3. DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*, std::shared_ptr<DB::IAST>&) @ 0x00000000138fc873
4. DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x00000000138fbba4
5. DB::TCPHandler::runImpl() @ 0x0000000014c2aaec
6. DB::TCPHandler::run() @ 0x0000000014c48cd9
7. Poco::Net::TCPServerConnection::start() @ 0x00000000183727c7
8. Poco::Net::TCPServerDispatcher::run() @ 0x0000000018372c19
9. Poco::PooledThread::run() @ 0x000000001833ef5b
10. Poco::ThreadImpl::runnableEntry(void*) @ 0x000000001833d43d
11. ? @ 0x00007f1fb820eac3
12. ? @ 0x00007f1fb829fa04

ParkGyeongTae · 2025-05-02T04:16:40Z

결과를 설명하는 부분에 마크다운 언어가 포함돼 안이쁘게 나오는 현상 수정.

ParkGyeongTae · 2025-05-02T04:17:21Z

리턴받는 문자열에 구분자가 포함되는 현상을 코드 출력으로 수정.

ehddnr301 · 2025-05-05T05:57:20Z

👍 아주 잘 동작합니다!
처음엔 LangChain의 output parser를 별도로 써야 하나 고민했는데, 이미 prompt변경없이도 충분히 원하는 형식으로 출력되고 있어서,
정규식을 활용해 안정적으로 output을 추출할 수 있게 된 것 같습니다! 너무 좋은 구현입니다!!

ParkGyeongTae · 2025-05-06T06:44:25Z

@ehddnr301 확인해주셔서 감사합니다~~!

fix: Context 에러 수정 및 Streamlit UI 파싱 수정, 웹페이지 클리닝

bd3b60d

ParkGyeongTae added this to the Mission: Version 1.0.0 Complete ✅ milestone May 2, 2025

ParkGyeongTae self-assigned this May 2, 2025

ehddnr301 approved these changes May 5, 2025

View reviewed changes

ehddnr301 mentioned this pull request May 5, 2025

State Output Format 설정 #98

Open

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix: SQL Context 에러 수정 및 Streamlit UI 파싱 수정, 웹페이지 클리닝 #91

fix: SQL Context 에러 수정 및 Streamlit UI 파싱 수정, 웹페이지 클리닝 #91

ParkGyeongTae commented May 2, 2025

ParkGyeongTae commented May 2, 2025

ParkGyeongTae commented May 2, 2025

ParkGyeongTae commented May 2, 2025

ParkGyeongTae commented May 2, 2025

ehddnr301 commented May 5, 2025

ParkGyeongTae commented May 6, 2025

fix: SQL Context 에러 수정 및 Streamlit UI 파싱 수정, 웹페이지 클리닝 #91

Are you sure you want to change the base?

fix: SQL Context 에러 수정 및 Streamlit UI 파싱 수정, 웹페이지 클리닝 #91

Conversation

ParkGyeongTae commented May 2, 2025

#️⃣ Issue Number

📝 요약(Summary)

💬 To Reviewers (선택)

PR Checklist

reference) How to Code Review

ParkGyeongTae commented May 2, 2025

ParkGyeongTae commented May 2, 2025

ParkGyeongTae commented May 2, 2025

ParkGyeongTae commented May 2, 2025

ehddnr301 commented May 5, 2025

ParkGyeongTae commented May 6, 2025