diff --git a/.codeboarding/Client_API.md b/.codeboarding/Client_API.md new file mode 100644 index 0000000000..9c1c62ac07 --- /dev/null +++ b/.codeboarding/Client_API.md @@ -0,0 +1,273 @@ +```mermaid + +graph LR + + Redis_Client_Synchronous_["Redis Client (Synchronous)"] + + Redis_Client_Asynchronous_["Redis Client (Asynchronous)"] + + Connection_Management["Connection Management"] + + Protocol_Handling["Protocol Handling"] + + Command_Implementations["Command Implementations"] + + Error_Handling["Error Handling"] + + Authentication_Security["Authentication & Security"] + + Backoff_Strategies["Backoff Strategies"] + + Event_Management["Event Management"] + + Redis_Client_Synchronous_ -- "uses" --> Connection_Management + + Redis_Client_Synchronous_ -- "uses" --> Command_Implementations + + Redis_Client_Synchronous_ -- "uses" --> Protocol_Handling + + Redis_Client_Synchronous_ -- "uses" --> Error_Handling + + Redis_Client_Asynchronous_ -- "uses" --> Connection_Management + + Redis_Client_Asynchronous_ -- "uses" --> Command_Implementations + + Redis_Client_Asynchronous_ -- "uses" --> Protocol_Handling + + Redis_Client_Asynchronous_ -- "uses" --> Error_Handling + + Connection_Management -- "uses" --> Protocol_Handling + + Connection_Management -- "uses" --> Authentication_Security + + Connection_Management -- "uses" --> Backoff_Strategies + + Connection_Management -- "dispatches events to" --> Event_Management + + Protocol_Handling -- "raises exceptions to" --> Error_Handling + + Command_Implementations -- "uses" --> Protocol_Handling + + Command_Implementations -- "raises errors to" --> Error_Handling + + Authentication_Security -- "raises errors to" --> Error_Handling + + Event_Management -- "triggers" --> Authentication_Security + + click Connection_Management href "https://github.com/redis/redis-py/blob/master/.codeboarding//Connection_Management.md" "Details" + +``` + + + +[![CodeBoarding](https://img.shields.io/badge/Generated%20by-CodeBoarding-9cf?style=flat-square)](https://github.com/CodeBoarding/GeneratedOnBoardings)[![Demo](https://img.shields.io/badge/Try%20our-Demo-blue?style=flat-square)](https://www.codeboarding.org/demo)[![Contact](https://img.shields.io/badge/Contact%20us%20-%20contact@codeboarding.org-lightgrey?style=flat-square)](mailto:contact@codeboarding.org) + + + +## Details + + + +Analysis of the Client API subsystem in redis-py + + + +### Redis Client (Synchronous) + +This component provides the synchronous, blocking interface for interacting with a Redis server. It exposes methods for all standard Redis commands, allowing users to execute operations directly. It also manages synchronous command pipelines and transactions. + + + + + +**Related Classes/Methods**: + + + +- `redis.client.Redis` (111:667) + +- `redis.client.Pipeline` (1278:1630) + + + + + +### Redis Client (Asynchronous) + +This component offers the asynchronous, non-blocking interface for interacting with Redis, built on `asyncio`. It mirrors the synchronous client's functionality but is designed for high-concurrency applications, enabling efficient I/O operations without blocking the event loop. + + + + + +**Related Classes/Methods**: + + + +- `redis.asyncio.client.Redis` (108:714) + +- `redis.asyncio.client.Pipeline` (1250:1617) + + + + + +### Connection Management [[Expand]](./Connection_Management.md) + +This component is responsible for establishing, maintaining, and pooling connections to Redis servers. It handles the low-level network communication details, including TCP/IP sockets, SSL/TLS, and Unix domain sockets, ensuring efficient resource utilization and connection stability. + + + + + +**Related Classes/Methods**: + + + +- `redis.connection.ConnectionPool` (1308:1647) + +- `redis.connection.Connection` (729:800) + +- `redis.asyncio.connection.ConnectionPool` (1030:1252) + +- `redis.asyncio.connection.Connection` (722:776) + + + + + +### Protocol Handling + +This component manages the serialization and deserialization of data according to the Redis Serialization Protocol (RESP). It encodes Python commands into RESP format for transmission to Redis and decodes RESP responses back into Python data types. It supports both RESP2 and RESP3 and can leverage `hiredis` for faster parsing. + + + + + +**Related Classes/Methods**: + + + +- `redis._parsers.base.BaseParser` (53:104) + +- `redis._parsers.hiredis._HiredisParser` (40:183) + +- `redis._parsers.resp2._RESP2Parser` (8:67) + +- `redis._parsers.resp3._RESP3Parser` (14:130) + + + + + +### Command Implementations + +This component provides the concrete implementations for the vast array of Redis commands. Commands are often organized into mixin classes (e.g., `CoreCommands`, `JSONCommands`, `SearchCommands`) that are then inherited by the main `Redis` client classes, promoting modularity and extensibility. + + + + + +**Related Classes/Methods**: + + + +- `redis.commands.core.CoreCommands` (6642:6655) + +- `redis.commands.core.AsyncCoreCommands` (6658:6671) + +- `redis.commands.json.JSON` (1:1) + +- `redis.commands.search.Search` (1:1) + + + + + +### Error Handling + +This component defines a comprehensive hierarchy of custom exception classes specific to Redis operations. It provides a structured and user-friendly way to report various issues, such as connection failures, invalid responses, authentication errors, and cluster-specific problems. + + + + + +**Related Classes/Methods**: + + + +- `redis.exceptions.RedisError` (3:4) + +- `redis.exceptions.ConnectionError` (7:8) + +- `redis.exceptions.ResponseError` (31:32) + + + + + +### Authentication & Security + +This component manages authentication credentials and secure communication settings. It handles mechanisms like password-based authentication, token-based authentication (e.g., JWT), and SSL/TLS configurations to ensure secure connections to Redis servers. + + + + + +**Related Classes/Methods**: + + + +- `redis.auth.token.TokenInterface` (6:29) + +- `redis.credentials.CredentialProvider` (7:20) + + + + + +### Backoff Strategies + +This component provides various algorithms for implementing retry logic with exponential backoff and jitter. It helps in gracefully handling transient network issues or temporary server unavailability by retrying failed operations with increasing delays, preventing overwhelming the server. + + + + + +**Related Classes/Methods**: + + + +- `redis.backoff.AbstractBackoff` (9:23) + +- `redis.backoff.ExponentialBackoff` (53:74) + + + + + +### Event Management + +This component provides a mechanism for dispatching and listening to internal events within the client library. It allows for custom logic to be triggered at specific points in the client's lifecycle, such as after a connection is released or during re-authentication, enabling extensibility and observability. + + + + + +**Related Classes/Methods**: + + + +- `redis.event.EventDispatcher` (56:89) + +- `redis.event.EventListenerInterface` (10:17) + + + + + + + + + +### [FAQ](https://github.com/CodeBoarding/GeneratedOnBoardings/tree/main?tab=readme-ov-file#faq) \ No newline at end of file diff --git a/.codeboarding/Command & Module Execution.md b/.codeboarding/Command & Module Execution.md new file mode 100644 index 0000000000..bf0c21554c --- /dev/null +++ b/.codeboarding/Command & Module Execution.md @@ -0,0 +1,379 @@ +```mermaid + +graph LR + + RedisCoreCommands["RedisCoreCommands"] + + RedisModuleIntegration["RedisModuleIntegration"] + + RedisJSONModule["RedisJSONModule"] + + RedisSearchModule["RedisSearchModule"] + + RedisTimeSeriesModule["RedisTimeSeriesModule"] + + RedisBloomModule["RedisBloomModule"] + + RedisVectorSetModule["RedisVectorSetModule"] + + RedisClusterManagement["RedisClusterManagement"] + + RedisScriptingAndPubSub["RedisScriptingAndPubSub"] + + RedisCommandHelpers["RedisCommandHelpers"] + + RedisModuleIntegration -- "integrates" --> RedisJSONModule + + RedisModuleIntegration -- "integrates" --> RedisSearchModule + + RedisModuleIntegration -- "integrates" --> RedisTimeSeriesModule + + RedisModuleIntegration -- "integrates" --> RedisBloomModule + + RedisModuleIntegration -- "integrates" --> RedisVectorSetModule + + RedisClusterManagement -- "utilizes" --> RedisCommandHelpers + + RedisClusterManagement -- "executes commands via" --> redis_typing_CommandsProtocol_execute_command + + RedisCoreCommands -- "executes commands via" --> redis_typing_CommandsProtocol_execute_command + + RedisCoreCommands -- "utilizes" --> RedisCommandHelpers + + RedisScriptingAndPubSub -- "executes commands via" --> redis_typing_CommandsProtocol_execute_command + + RedisScriptingAndPubSub -- "utilizes" --> RedisCommandHelpers + + RedisVectorSetModule -- "executes commands via" --> redis_typing_CommandsProtocol_execute_command + + RedisVectorSetModule -- "utilizes" --> RedisCommandHelpers + + RedisTimeSeriesModule -- "executes commands via" --> redis_typing_CommandsProtocol_execute_command + + RedisTimeSeriesModule -- "utilizes" --> RedisCommandHelpers + + RedisSearchModule -- "executes commands via" --> redis_typing_CommandsProtocol_execute_command + + RedisSearchModule -- "utilizes" --> RedisCommandHelpers + + RedisBloomModule -- "executes commands via" --> redis_typing_CommandsProtocol_execute_command + + RedisBloomModule -- "utilizes" --> RedisCommandHelpers + + RedisJSONModule -- "executes commands via" --> redis_typing_CommandsProtocol_execute_command + + RedisJSONModule -- "utilizes" --> RedisCommandHelpers + +``` + +[![CodeBoarding](https://img.shields.io/badge/Generated%20by-CodeBoarding-9cf?style=flat-square)](https://github.com/CodeBoarding/GeneratedOnBoardings)[![Demo](https://img.shields.io/badge/Try%20our-Demo-blue?style=flat-square)](https://www.codeboarding.org/demo)[![Contact](https://img.shields.io/badge/Contact%20us%20-%20contact@codeboarding.org-lightgrey?style=flat-square)](mailto:contact@codeboarding.org) + + + +## Component Details + + + +This component overview describes the structure, flow, and purpose of the Command & Module Execution subsystem within the Redis client library. It focuses on how standard Redis commands are implemented and executed, and how interactions with various Redis Modules (e.g., JSON, Search, TimeSeries) are facilitated. The system provides a comprehensive interface for both core Redis functionalities and extended module capabilities, ensuring efficient data operations and server management. + + + +### RedisCoreCommands + +This component provides the fundamental Redis commands for interacting with various data structures (strings, lists, sets, sorted sets, hashes, streams, geo-spatial) and handles administrative and management commands for the Redis server (configuration, client management, server-wide operations, ACL, modules, cluster commands). It serves as the primary interface for basic Redis operations. + + + + + +**Related Classes/Methods**: + + + +- `redis.commands.core.ACLCommands` (full file reference) + +- `redis.commands.core.ManagementCommands` (full file reference) + +- `redis.commands.core.AsyncManagementCommands` (full file reference) + +- `redis.commands.core.BitFieldOperation` (full file reference) + +- `redis.commands.core.BasicKeyCommands` (full file reference) + +- `redis.commands.core.AsyncBasicKeyCommands` (full file reference) + +- `redis.commands.core.ListCommands` (full file reference) + +- `redis.commands.core.ScanCommands` (full file reference) + +- `redis.commands.core.AsyncScanCommands` (full file reference) + +- `redis.commands.core.SetCommands` (full file reference) + +- `redis.commands.core.StreamCommands` (full file reference) + +- `redis.commands.core.SortedSetCommands` (full file reference) + +- `redis.commands.core.HyperlogCommands` (full file reference) + +- `redis.commands.core.HashCommands` (full file reference) + +- `redis.commands.core.GeoCommands` (full file reference) + +- `redis.commands.core.ModuleCommands` (full file reference) + +- `redis.commands.core.AsyncModuleCommands` (full file reference) + +- `redis.commands.core.ClusterCommands` (full file reference) + + + + + +### RedisModuleIntegration + +This component acts as an integration layer for various Redis modules, providing a unified interface to access their functionalities. It serves as a central point for dispatching commands to specific module clients. + + + + + +**Related Classes/Methods**: + + + +- `redis.commands.redismodules.RedisModuleCommands` (14:91) + +- `redis.commands.redismodules.AsyncRedisModuleCommands` (94:101) + + + + + +### RedisJSONModule + +This component offers a client for the RedisJSON module, allowing for efficient storage and manipulation of JSON documents within Redis. It provides methods for JSON-specific operations like setting, getting, and manipulating JSON paths. + + + + + +**Related Classes/Methods**: + + + +- `redis.commands.json.commands.JSONCommands` (13:431) + +- `redis.commands.json.JSON` (full file reference) + + + + + +### RedisSearchModule + +This component provides a comprehensive client for the Redis Search module, supporting index creation, document management, complex query building, and aggregation. It enables full-text search capabilities within Redis. + + + + + +**Related Classes/Methods**: + + + +- `redis.commands.search.index_definition.IndexDefinition` (11:79) + +- `redis.commands.search.aggregation.AggregateRequest` (89:372) + +- `redis.commands.search.field.TextField` (79:109) + +- `redis.commands.search.field.NumericField` (112:118) + +- `redis.commands.search.field.GeoShapeField` (121:133) + +- `redis.commands.search.field.GeoField` (136:142) + +- `redis.commands.search.field.TagField` (145:168) + +- `redis.commands.search.field.VectorField` (171:210) + +- `redis.commands.search.commands.SearchCommands` (full file reference) + +- `redis.commands.search.commands.AsyncSearchCommands` (full file reference) + +- `redis.commands.search.query.Query` (6:339) + +- `redis.commands.search.query.NumericFilter` (347:364) + +- `redis.commands.search.query.GeoFilter` (367:376) + +- `redis.commands.search.querystring` (316:317) + +- `redis.commands.search.reducers` (full file reference) + +- `redis.commands.search.suggestion.Suggestion` (6:20) + +- `redis.commands.search.suggestion.SuggestionParser` (23:55) + +- `redis.commands.search.result.Result` (7:87) + +- `redis.commands.search.Search` (full file reference) + +- `redis.commands.search.AsyncSearch` (full file reference) + + + + + +### RedisTimeSeriesModule + +This component offers a client for the Redis TimeSeries module, allowing for the creation, manipulation, and querying of time-series data. It provides functionalities for adding samples, querying ranges, and managing time-series data. + + + + + +**Related Classes/Methods**: + + + +- `redis.commands.timeseries.info.TSInfo` (5:91) + +- `redis.commands.timeseries.commands.TimeSeriesCommands` (25:1000) + +- `redis.commands.timeseries.TimeSeries` (full file reference) + + + + + +### RedisBloomModule + +This component provides client-side access to the RedisBloom module, enabling the use of probabilistic data structures like Bloom filters, Cuckoo filters, Count-Min sketches, and TopK. It offers methods for interacting with these specialized data structures. + + + + + +**Related Classes/Methods**: + + + +- `redis.commands.bf.info.BFInfo` (4:26) + +- `redis.commands.bf.info.CFInfo` (29:57) + +- `redis.commands.bf.info.TDigestInfo` (92:120) + +- `redis.commands.bf.commands.TOPKCommands` (292:356) + +- `redis.commands.bf.CMSBloom` (full file reference) + +- `redis.commands.bf.TOPKBloom` (full file reference) + +- `redis.commands.bf.CFBloom` (full file reference) + +- `redis.commands.bf.TDigestBloom` (full file reference) + +- `redis.commands.bf.BFBloom` (full file reference) + + + + + +### RedisVectorSetModule + +This component provides the client-side interface for interacting with the Redis VectorSet module, enabling vector similarity search and related operations. It allows for storing and querying vector data within Redis. + + + + + +**Related Classes/Methods**: + + + +- `redis.commands.vectorset.commands.VectorSetCommands` (40:367) + +- `redis.commands.vectorset.VectorSet` (full file reference) + + + + + +### RedisClusterManagement + +This component is responsible for managing Redis Cluster specific operations, including multi-key commands, node management, and data partitioning across slots. It provides an interface for interacting with a Redis Cluster setup. + + + + + +**Related Classes/Methods**: + + + +- `redis.commands.cluster.ClusterMultiKeyCommands` (99:260) + +- `redis.commands.cluster.AsyncClusterMultiKeyCommands` (263:339) + +- `redis.commands.cluster.ClusterManagementCommands` (342:692) + +- `redis.commands.cluster.AsyncClusterManagementCommands` (695:719) + +- `redis.commands.cluster.ClusterDataAccessCommands` (722:810) + + + + + +### RedisScriptingAndPubSub + +This component encapsulates functionalities related to Redis scripting (Lua scripts) and Publish/Subscribe messaging. It provides methods for executing scripts, managing script caches, and handling pub/sub operations. + + + + + +**Related Classes/Methods**: + + + +- `redis.commands.core.Script` (full file reference) + +- `redis.commands.core.PubSubCommands` (full file reference) + +- `redis.commands.core.ScriptCommands` (full file reference) + +- `redis.commands.core.AsyncScriptCommands` (full file reference) + +- `redis.commands.core.FunctionCommands` (full file reference) + + + + + +### RedisCommandHelpers + +This component provides various utility functions and helper methods that are commonly used across different Redis command implementations for tasks like argument parsing, data conversion, and protocol version handling. It supports the other command components by offering shared functionalities. + + + + + +**Related Classes/Methods**: + + + +- `redis.commands.helpers` (full file reference) + + + + + + + + + +### [FAQ](https://github.com/CodeBoarding/GeneratedOnBoardings/tree/main?tab=readme-ov-file#faq) \ No newline at end of file diff --git a/.codeboarding/Connection & Protocol Management.md b/.codeboarding/Connection & Protocol Management.md new file mode 100644 index 0000000000..26ee5c68c3 --- /dev/null +++ b/.codeboarding/Connection & Protocol Management.md @@ -0,0 +1,347 @@ +```mermaid + +graph LR + + Core_Connection_Management["Core Connection Management"] + + Connection_Pooling["Connection Pooling"] + + Protocol_Parsers["Protocol Parsers"] + + Authentication_Security["Authentication & Security"] + + Error_Handling["Error Handling"] + + Client_Interfaces["Client Interfaces"] + + Utility_Eventing["Utility & Eventing"] + + Retry_Backoff["Retry & Backoff"] + + Caching["Caching"] + + Client_Interfaces -- "uses" --> Connection_Pooling + + Connection_Pooling -- "manages" --> Core_Connection_Management + + Core_Connection_Management -- "uses" --> Protocol_Parsers + + Core_Connection_Management -- "raises" --> Error_Handling + + Core_Connection_Management -- "integrates" --> Authentication_Security + + Core_Connection_Management -- "applies" --> Retry_Backoff + + Protocol_Parsers -- "raises" --> Error_Handling + + Authentication_Security -- "raises" --> Error_Handling + + Connection_Pooling -- "notifies" --> Utility_Eventing + + Connection_Pooling -- "leverages" --> Caching + + Client_Interfaces -- "utilizes" --> Caching + + Utility_Eventing -- "supports" --> Core_Connection_Management + + Utility_Eventing -- "supports" --> Protocol_Parsers + +``` + +[![CodeBoarding](https://img.shields.io/badge/Generated%20by-CodeBoarding-9cf?style=flat-square)](https://github.com/CodeBoarding/GeneratedOnBoardings)[![Demo](https://img.shields.io/badge/Try%20our-Demo-blue?style=flat-square)](https://www.codeboarding.org/demo)[![Contact](https://img.shields.io/badge/Contact%20us%20-%20contact@codeboarding.org-lightgrey?style=flat-square)](mailto:contact@codeboarding.org) + + + +## Component Details + + + +The Connection & Protocol Management subsystem in the Redis client library is a comprehensive system designed to facilitate robust and efficient communication with Redis servers. It manages the entire lifecycle of connections, from establishment and pooling to secure authentication and error handling. The subsystem handles the intricacies of Redis Serialization Protocols (RESP2 and RESP3) for command encoding and response parsing, ensuring data integrity and performance. It also incorporates caching mechanisms, retry strategies for transient failures, and provides high-level client interfaces that abstract the underlying complexities, enabling seamless interaction for users. + + + +### Core Connection Management + +This component encapsulates the fundamental logic for establishing, maintaining, and managing connections to Redis servers, including both synchronous and asynchronous implementations. It handles various connection types such as standard TCP, SSL, and Unix domain sockets, and manages the lifecycle of individual connections. + + + + + +**Related Classes/Methods**: + + + +- `redis.connection.AbstractConnection` (full file reference) + +- `redis.connection.Connection` (full file reference) + +- `redis.connection.SSLConnection` (full file reference) + +- `redis.connection.UnixDomainSocketConnection` (full file reference) + +- `redis.connection.CacheProxyConnection` (full file reference) + +- `redis.asyncio.connection.AbstractConnection` (106:720) + +- `redis.asyncio.connection.Connection` (723:777) + +- `redis.asyncio.connection.SSLConnection` (780:844) + +- `redis.asyncio.connection.UnixDomainSocketConnection` (916:937) + + + + + +### Connection Pooling + +This component is responsible for efficiently managing a pool of Redis connections. It optimizes resource utilization by reusing existing connections, thereby reducing the overhead of establishing new connections for each operation. It provides mechanisms for acquiring and releasing connections, and supports both blocking and non-blocking behaviors. + + + + + +**Related Classes/Methods**: + + + +- `redis.connection.ConnectionPool` (full file reference) + +- `redis.connection.BlockingConnectionPool` (full file reference) + +- `redis.asyncio.connection.ConnectionPool` (full file reference) + +- `redis.asyncio.connection.BlockingConnectionPool` (full file reference) + + + + + +### Protocol Parsers + +This component handles the serialization of commands sent to Redis and the deserialization of responses received from the server. It supports different Redis Serialization Protocols (RESP2 and RESP3) and can leverage optimized C implementations like hiredis for improved performance. It also includes logic for parsing command structures and handling socket buffer operations. + + + + + +**Related Classes/Methods**: + + + +- `redis._parsers.encoders.Encoder` (4:44) + +- `redis._parsers.socket.SocketBuffer` (29:162) + +- `redis._parsers.commands.AbstractCommandsParser` (10:53) + +- `redis._parsers.commands.CommandsParser` (56:170) + +- `redis._parsers.commands.AsyncCommandsParser` (173:281) + +- `redis._parsers.hiredis._HiredisParser` (41:184) + +- `redis._parsers.hiredis._AsyncHiredisParser` (187:295) + +- `redis._parsers.resp3._RESP3Parser` (15:131) + +- `redis._parsers.resp3._AsyncRESP3Parser` (134:257) + +- `redis._parsers.base.BaseParser` (54:105) + +- `redis._parsers.base._RESPBase` (108:140) + +- `redis._parsers.base._AsyncRESPBase` (213:289) + +- `redis._parsers.resp2._RESP2Parser` (9:68) + +- `redis._parsers.resp2._AsyncRESP2Parser` (71:132) + + + + + +### Authentication & Security + +This component provides functionalities for authenticating with Redis servers and ensuring secure communication. It includes mechanisms for managing authentication tokens, such as JWT, and implements OCSP (Online Certificate Status Protocol) verification to validate SSL certificates, enhancing the security posture of connections. + + + + + +**Related Classes/Methods**: + + + +- `redis.auth.token_manager.TokenManager` (121:340) + +- `redis.auth.token.SimpleToken` (44:75) + +- `redis.auth.token.JWToken` (78:130) + +- `redis.ocsp._verify_response` (22:47) + +- `redis.ocsp._check_certificate` (50:106) + +- `redis.ocsp._get_certificates` (109:123) + +- `redis.ocsp.ocsp_staple_verifier` (142:167) + +- `redis.ocsp.OCSPVerifier` (170:308) + +- `redis.asyncio.connection.RedisSSLContext` (847:913) + + + + + +### Error Handling + +This component defines a comprehensive set of custom exception classes that represent various error conditions encountered during Redis operations, such as connection failures, timeouts, authentication issues, and invalid responses. It centralizes error management, allowing for more specific and robust error handling throughout the library. + + + + + +**Related Classes/Methods**: + + + +- `redis.exceptions.DataError` (36:37) + +- `redis.exceptions.ConnectionError` (8:9) + +- `redis.exceptions.TimeoutError` (12:13) + +- `redis.exceptions.AuthenticationError` (16:17) + +- `redis.exceptions.RedisError` (4:5) + +- `redis.exceptions.InvalidResponse` (28:29) + +- `redis.exceptions.ResponseError` (32:33) + +- `redis.auth.err.TokenRenewalErr` (25:31) + +- `redis.auth.err.InvalidTokenSchemaErr` (13:22) + +- `redis.exceptions.AuthorizationError` (20:21) + + + + + +### Client Interfaces + +This component provides the high-level API for users to interact with Redis. It includes the standard Redis client, as well as specialized clients for Redis Cluster and Redis Sentinel, abstracting the underlying connection and protocol details to offer a user-friendly interface for executing Redis commands. + + + + + +**Related Classes/Methods**: + + + +- `redis.cluster.RedisCluster` (456:1000) + +- `redis.cluster.NodesManager` (full file reference) + +- `redis.sentinel.SentinelConnectionPool` (137:195) + +- `redis.client.Redis` (112:670) + + + + + +### Utility & Eventing + +This component comprises a collection of general-purpose utility functions that support various operations across the library, such as string manipulation, version comparison, and argument deprecation handling. It also includes an event dispatching mechanism that allows different parts of the system to communicate and react to specific events, like connection releases. + + + + + +**Related Classes/Methods**: + + + +- `redis.utils.get_lib_version` (211:216) + +- `redis.utils.format_error_message` (219:228) + +- `redis.utils.str_if_bytes` (60:63) + +- `redis.utils.deprecated_args` (153:195) + +- `redis.utils.ensure_string` (261:267) + +- `redis.utils.compare_versions` (231:258) + +- `redis.event.EventDispatcher` (57:90) + +- `redis.event.AfterConnectionReleasedEvent` (93:103) + +- `redis.event.AsyncAfterConnectionReleasedEvent` (106:107) + + + + + +### Retry & Backoff + +This component implements robust strategies for handling transient failures by retrying operations with configurable delays. It includes mechanisms for exponential backoff and allows for the definition of supported error types, ensuring that the client can gracefully recover from temporary network issues or server unavailability. + + + + + +**Related Classes/Methods**: + + + +- `redis.retry.Retry` (13:95) + +- `redis.backoff.NoBackoff` (47:51) + +- `redis.asyncio.retry.Retry` (13:79) + + + + + +### Caching + +This component provides an internal caching mechanism used to store and retrieve frequently accessed data, such as command information or connection details, to improve performance. It includes different caching policies, like LRU, and a factory for creating cache instances. + + + + + +**Related Classes/Methods**: + + + +- `redis.cache.DefaultCache` (142:224) + +- `redis.cache.LRUPolicy` (227:271) + +- `redis.cache.CacheFactory` (392:401) + +- `redis.cache.CacheConfig` (278:383) + +- `redis.cache.CacheKey` (19:21) + +- `redis.cache.CacheEntry` (24:43) + + + + + + + + + +### [FAQ](https://github.com/CodeBoarding/GeneratedOnBoardings/tree/main?tab=readme-ov-file#faq) \ No newline at end of file diff --git a/.codeboarding/Connection_Management.md b/.codeboarding/Connection_Management.md new file mode 100644 index 0000000000..f6cc50977c --- /dev/null +++ b/.codeboarding/Connection_Management.md @@ -0,0 +1,179 @@ +```mermaid + +graph LR + + Connection_Abstraction["Connection Abstraction"] + + Connection_Pooling["Connection Pooling"] + + Sentinel_Integration["Sentinel Integration"] + + Cluster_Integration["Cluster Integration"] + + Exception_Handling["Exception Handling"] + + Connection_Abstraction -- "uses" --> Exception_Handling + + Connection_Pooling -- "uses" --> Connection_Abstraction + + Sentinel_Integration -- "relies on" --> Connection_Pooling + + Sentinel_Integration -- "uses" --> Exception_Handling + + Cluster_Integration -- "relies on" --> Connection_Pooling + + Cluster_Integration -- "uses" --> Exception_Handling + +``` + + + +[![CodeBoarding](https://img.shields.io/badge/Generated%20by-CodeBoarding-9cf?style=flat-square)](https://github.com/CodeBoarding/GeneratedOnBoardings)[![Demo](https://img.shields.io/badge/Try%20our-Demo-blue?style=flat-square)](https://www.codeboarding.org/demo)[![Contact](https://img.shields.io/badge/Contact%20us%20-%20contact@codeboarding.org-lightgrey?style=flat-square)](mailto:contact@codeboarding.org) + + + +## Details + + + +The Connection Management subsystem in redis-py is crucial for establishing and maintaining connections to various Redis deployments. It abstracts the complexities of network communication, connection pooling, and handling different Redis topologies (standalone, Sentinel, Cluster). + + + +### Connection Abstraction + +This component provides the foundational classes for managing individual connections to a Redis server. It handles the low-level details of establishing TCP or Unix socket connections, sending commands, and receiving responses. It also includes support for SSL/TLS connections. + + + + + +**Related Classes/Methods**: + + + +- `redis.connection.Connection` (729:800) + +- `redis.connection.SSLConnection` (1017:1185) + +- `redis.connection.UnixDomainSocketConnection` (1188:1220) + +- `redis.asyncio.connection.Connection` (722:776) + +- `redis.asyncio.connection.SSLConnection` (779:843) + +- `redis.asyncio.connection.UnixDomainSocketConnection` (915:936) + + + + + +### Connection Pooling + +This component manages a pool of connections to Redis, improving performance by reusing existing connections instead of establishing new ones for each operation. It handles connection lifecycle, including creation, release, and optional blocking behavior when the pool is exhausted. + + + + + +**Related Classes/Methods**: + + + +- `redis.connection.ConnectionPool` (1308:1647) + +- `redis.connection.BlockingConnectionPool` (1650:1817) + +- `redis.asyncio.connection.ConnectionPool` (1030:1252) + +- `redis.asyncio.connection.BlockingConnectionPool` (1255:1332) + + + + + +### Sentinel Integration + +This component provides support for connecting to and managing Redis Sentinel deployments. It enables the client to discover the current master and replica nodes, handle failovers, and automatically re-route commands to the correct master. + + + + + +**Related Classes/Methods**: + + + +- `redis.sentinel.Sentinel` (197:420) + +- `redis.sentinel.SentinelConnectionPool` (136:194) + +- `redis.sentinel.SentinelManagedConnection` (19:81) + +- `redis.asyncio.sentinel.Sentinel` (166:400) + +- `redis.asyncio.sentinel.SentinelConnectionPool` (95:163) + +- `redis.asyncio.sentinel.SentinelManagedConnection` (25:88) + + + + + +### Cluster Integration + +This component handles connections and command routing within a Redis Cluster environment. It manages slot mapping, handles redirection errors (MOVED/ASK), and discovers cluster nodes, ensuring commands are sent to the appropriate shard. + + + + + +**Related Classes/Methods**: + + + +- `redis.cluster.RedisCluster` (455:1358) + +- `redis.cluster.NodesManager` (1441:1861) + +- `redis.asyncio.cluster.RedisCluster` (98:988) + +- `redis.asyncio.cluster.NodesManager` (1210:1517) + + + + + +### Exception Handling + +This component defines a hierarchy of custom exceptions specific to Redis operations and connection issues. It provides a structured way to handle errors such as connection failures, timeouts, authentication errors, and Redis-specific response errors. + + + + + +**Related Classes/Methods**: + + + +- `redis.exceptions.RedisError` (3:4) + +- `redis.exceptions.ConnectionError` (7:8) + +- `redis.exceptions.TimeoutError` (11:12) + +- `redis.exceptions.AuthenticationError` (15:16) + +- `redis.exceptions.ResponseError` (31:32) + +- `redis.exceptions.ClusterError` (119:125) + + + + + + + + + +### [FAQ](https://github.com/CodeBoarding/GeneratedOnBoardings/tree/main?tab=readme-ov-file#faq) \ No newline at end of file diff --git a/.codeboarding/Error Handling.md b/.codeboarding/Error Handling.md new file mode 100644 index 0000000000..26e3654a7b --- /dev/null +++ b/.codeboarding/Error Handling.md @@ -0,0 +1,371 @@ +```mermaid + +graph LR + + Error_Handling["Error Handling"] + + Redis_Client["Redis Client"] + + Connection_Management["Connection Management"] + + Redis_Cluster_Management["Redis Cluster Management"] + + Distributed_Locking["Distributed Locking"] + + Redis_Sentinel_Integration["Redis Sentinel Integration"] + + Protocol_Parsing["Protocol Parsing"] + + Authentication_and_Security["Authentication and Security"] + + Redis_Client -- "utilizes" --> Connection_Management + + Redis_Client -- "interprets responses via" --> Protocol_Parsing + + Redis_Client -- "raises" --> Error_Handling + + Connection_Management -- "relies on" --> Protocol_Parsing + + Connection_Management -- "raises" --> Error_Handling + + Redis_Cluster_Management -- "manages connections through" --> Connection_Management + + Redis_Cluster_Management -- "raises" --> Error_Handling + + Distributed_Locking -- "leverages" --> Redis_Client + + Distributed_Locking -- "raises" --> Error_Handling + + Redis_Sentinel_Integration -- "manages connections through" --> Connection_Management + + Redis_Sentinel_Integration -- "raises" --> Error_Handling + + Protocol_Parsing -- "raises" --> Error_Handling + + Authentication_and_Security -- "configures" --> Connection_Management + +``` + +[![CodeBoarding](https://img.shields.io/badge/Generated%20by-CodeBoarding-9cf?style=flat-square)](https://github.com/CodeBoarding/GeneratedOnBoardings)[![Demo](https://img.shields.io/badge/Try%20our-Demo-blue?style=flat-square)](https://www.codeboarding.org/demo)[![Contact](https://img.shields.io/badge/Contact%20us%20-%20contact@codeboarding.org-lightgrey?style=flat-square)](mailto:contact@codeboarding.org) + + + +## Component Details + + + +This architecture overview describes the core components of the redis-py library, focusing on how it manages connections, handles Redis commands, interacts with Redis clusters and Sentinels, and provides robust error handling and security features. The main flow involves clients initiating connections through connection management, sending commands which are then parsed and executed, and receiving responses that are processed by the protocol parsing component. Error handling is integrated throughout, providing specific exceptions for various operational failures. Asynchronous capabilities are seamlessly integrated into the core components, offering non-blocking operations for high-performance applications. + + + +### Error Handling + +Defines and manages custom exception classes for various Redis-related errors, providing a structured and specific way to handle different error scenarios that can occur during client-server interactions. + + + + + +**Related Classes/Methods**: + + + +- `redis.exceptions.RedisError` (4:5) + +- `redis.exceptions.ConnectionError` (8:9) + +- `redis.exceptions.AuthenticationError` (16:17) + +- `redis.exceptions.AuthorizationError` (20:21) + +- `redis.exceptions.BusyLoadingError` (24:25) + +- `redis.exceptions.MaxConnectionsError` (223:223) + +- `redis.exceptions.TimeoutError` (12:13) + +- `redis.exceptions.DataError` (36:37) + +- `redis.exceptions.PubSubError` (40:41) + +- `redis.exceptions.InvalidResponse` (28:29) + +- `redis.exceptions.ResponseError` (32:33) + +- `redis.exceptions.AuthenticationWrongNumberOfArgsError` (103:109) + +- `redis.exceptions.NoPermissionError` (72:73) + +- `redis.exceptions.ExecAbortError` (64:65) + +- `redis.exceptions.ReadOnlyError` (68:69) + +- `redis.exceptions.NoScriptError` (48:49) + +- `redis.exceptions.OutOfMemoryError` (52:61) + +- `redis.exceptions.ModuleError` (76:77) + +- `redis.exceptions.WatchError` (44:45) + +- `redis.exceptions.LockError` (80:88) + +- `redis.exceptions.LockNotOwnedError` (91:94) + +- `redis.exceptions.TryAgainError` (171:179) + +- `redis.exceptions.ClusterError` (120:126) + +- `redis.exceptions.ClusterDownError` (129:142) + +- `redis.exceptions.MasterDownError` (202:208) + +- `redis.exceptions.AskError` (145:168) + +- `redis.exceptions.MovedError` (192:199) + +- `redis.exceptions.ClusterCrossSlotError` (182:189) + +- `redis.exceptions.RedisClusterException` (112:117) + +- `redis.exceptions.SlotNotCoveredError` (211:220) + +- `redis.exceptions.CrossSlotTransactionError` (226:232) + +- `redis.exceptions.InvalidPipelineStack` (235:241) + + + + + +### Redis Client + +Provides the fundamental synchronous and asynchronous interfaces for interacting with a single Redis instance. It includes functionalities for executing commands, managing Pub/Sub subscriptions, and handling transactional pipelines. + + + + + +**Related Classes/Methods**: + + + +- `redis.client.Redis` (112:670) + +- `redis.client.Monitor` (676:740) + +- `redis.client.PubSub` (743:1000) + +- `redis.client.Pipeline` (full file reference) + +- `redis.asyncio.client.Redis` (full file reference) + +- `redis.asyncio.client.Monitor` (full file reference) + +- `redis.asyncio.client.PubSub` (full file reference) + +- `redis.asyncio.client.Pipeline` (full file reference) + + + + + +### Connection Management + +Responsible for establishing, maintaining, and pooling synchronous and asynchronous connections to Redis servers. It handles connection health checks, error handling during connection attempts, and SSL/TLS wrapping. + + + + + +**Related Classes/Methods**: + + + +- `redis.connection.HiredisRespSerializer` (full file reference) + +- `redis.connection.AbstractConnection` (full file reference) + +- `redis.connection.CacheProxyConnection` (full file reference) + +- `redis.connection.SSLConnection` (full file reference) + +- `redis.connection.ConnectionPool` (full file reference) + +- `redis.connection.BlockingConnectionPool` (full file reference) + +- `redis.asyncio.connection.AbstractConnection` (106:720) + +- `redis.asyncio.connection.SSLConnection` (780:844) + +- `redis.asyncio.connection.RedisSSLContext` (847:913) + +- `redis.asyncio.connection.ConnectionPool` (full file reference) + +- `redis.asyncio.connection.BlockingConnectionPool` (full file reference) + + + + + +### Redis Cluster Management + +Manages interactions with a Redis Cluster for both synchronous and asynchronous operations. It handles node discovery, slot mapping, command routing, and error handling specific to a clustered environment, including transaction and pipeline strategies within the cluster. + + + + + +**Related Classes/Methods**: + + + +- `redis.cluster.RedisCluster` (456:1000) + +- `redis.cluster.NodesManager` (full file reference) + +- `redis.cluster.ClusterPubSub` (full file reference) + +- `redis.cluster.AbstractStrategy` (full file reference) + +- `redis.cluster.PipelineStrategy` (full file reference) + +- `redis.cluster.TransactionStrategy` (full file reference) + +- `redis.commands.cluster.ClusterManagementCommands` (342:692) + +- `redis.asyncio.cluster.RedisCluster` (full file reference) + +- `redis.asyncio.cluster.ClusterNode` (full file reference) + +- `redis.asyncio.cluster.NodesManager` (full file reference) + +- `redis.asyncio.cluster.PipelineStrategy` (full file reference) + +- `redis.asyncio.cluster.TransactionStrategy` (full file reference) + + + + + +### Distributed Locking + +Provides mechanisms for implementing distributed locks using Redis, ensuring atomicity and proper release of locks for both synchronous and asynchronous contexts. + + + + + +**Related Classes/Methods**: + + + +- `redis.lock.Lock` (14:343) + +- `redis.asyncio.lock.Lock` (17:334) + + + + + +### Redis Sentinel Integration + +Facilitates interaction with Redis Sentinel for high availability, allowing clients to discover master and slave nodes and handle failovers for both synchronous and asynchronous operations. + + + + + +**Related Classes/Methods**: + + + +- `redis.sentinel.SentinelManagedConnection` (20:82) + +- `redis.sentinel.SentinelConnectionPoolProxy` (89:134) + +- `redis.sentinel.Sentinel` (198:410) + +- `redis.asyncio.sentinel.SentinelManagedConnection` (26:89) + +- `redis.asyncio.sentinel.SentinelConnectionPool` (96:164) + +- `redis.asyncio.sentinel.Sentinel` (167:389) + + + + + +### Protocol Parsing + +Responsible for parsing responses from the Redis server according to the RESP protocol, including handling different RESP versions and error responses for both synchronous and asynchronous contexts. + + + + + +**Related Classes/Methods**: + + + +- `redis._parsers.encoders.Encoder` (4:44) + +- `redis._parsers.socket.SocketBuffer` (29:162) + +- `redis._parsers.commands.CommandsParser` (56:170) + +- `redis._parsers.commands.AsyncCommandsParser` (173:281) + +- `redis._parsers.hiredis._HiredisParser` (41:184) + +- `redis._parsers.hiredis._AsyncHiredisParser` (187:295) + +- `redis._parsers.resp3._RESP3Parser` (15:131) + +- `redis._parsers.resp3._AsyncRESP3Parser` (134:257) + +- `redis._parsers.base.BaseParser` (54:105) + +- `redis._parsers.base._AsyncRESPBase` (213:289) + +- `redis._parsers.resp2._RESP2Parser` (9:68) + +- `redis._parsers.resp2._AsyncRESP2Parser` (71:132) + + + + + +### Authentication and Security + +Handles authentication mechanisms, including token management and OCSP verification for secure connections. + + + + + +**Related Classes/Methods**: + + + +- `redis.ocsp._verify_response` (22:47) + +- `redis.ocsp._check_certificate` (50:106) + +- `redis.ocsp.ocsp_staple_verifier` (142:167) + +- `redis.ocsp.OCSPVerifier` (170:308) + +- `redis.auth.token_manager.TokenManager` (121:340) + +- `redis.auth.token.JWToken` (78:130) + + + + + + + + + +### [FAQ](https://github.com/CodeBoarding/GeneratedOnBoardings/tree/main?tab=readme-ov-file#faq) \ No newline at end of file diff --git a/.codeboarding/High Availability & Cluster Management.md b/.codeboarding/High Availability & Cluster Management.md new file mode 100644 index 0000000000..eb44e6abf8 --- /dev/null +++ b/.codeboarding/High Availability & Cluster Management.md @@ -0,0 +1,305 @@ +```mermaid + +graph LR + + Redis_Cluster_Client["Redis Cluster Client"] + + Cluster_Node_Manager["Cluster Node Manager"] + + Cluster_Node_Representation["Cluster Node Representation"] + + Load_Balancer["Load Balancer"] + + Cluster_Pub_Sub["Cluster Pub/Sub"] + + Cluster_Pipeline["Cluster Pipeline"] + + Pipeline_Strategy["Pipeline Strategy"] + + Transaction_Strategy["Transaction Strategy"] + + Redis_Sentinel_Client["Redis Sentinel Client"] + + Sentinel_Connection_Pool["Sentinel Connection Pool"] + + Redis_Cluster_Client -- "manages" --> Cluster_Node_Manager + + Redis_Cluster_Client -- "creates" --> Cluster_Pipeline + + Redis_Cluster_Client -- "creates" --> Cluster_Pub_Sub + + Cluster_Node_Manager -- "provides nodes to" --> Redis_Cluster_Client + + Cluster_Node_Manager -- "manages" --> Cluster_Node_Representation + + Cluster_Node_Manager -- "uses" --> Load_Balancer + + Cluster_Pipeline -- "delegates execution to" --> Pipeline_Strategy + + Cluster_Pipeline -- "delegates execution to" --> Transaction_Strategy + + Redis_Sentinel_Client -- "uses" --> Sentinel_Connection_Pool + + Sentinel_Connection_Pool -- "provides connections to" --> Redis_Sentinel_Client + +``` + +[![CodeBoarding](https://img.shields.io/badge/Generated%20by-CodeBoarding-9cf?style=flat-square)](https://github.com/CodeBoarding/GeneratedOnBoardings)[![Demo](https://img.shields.io/badge/Try%20our-Demo-blue?style=flat-square)](https://www.codeboarding.org/demo)[![Contact](https://img.shields.io/badge/Contact%20us%20-%20contact@codeboarding.org-lightgrey?style=flat-square)](mailto:contact@codeboarding.org) + + + +## Component Details + + + +This graph illustrates the architecture of the High Availability & Cluster Management subsystem, focusing on how Redis Cluster and Redis Sentinel functionalities are provided. It details the components responsible for node discovery, command routing, pipeline execution, and high-availability management in both cluster and sentinel environments. + + + +### Redis Cluster Client + +The Redis Cluster Client (`RedisCluster`) serves as the primary interface for applications to interact with a Redis Cluster. It handles command routing to appropriate nodes, manages retries on MOVED/ASK errors, and provides methods for obtaining pipeline and pub/sub instances. It supports both synchronous and asynchronous operations. + + + + + +**Related Classes/Methods**: + + + +- `redis.cluster.RedisCluster` (456:1000) + +- `redis.cluster.RedisCluster:__init__` (513:712) + +- `redis.cluster.RedisCluster:execute_command` (full file reference) + +- `redis.cluster.RedisCluster:pipeline` (841:866) + +- `redis.cluster.RedisCluster:pubsub` (834:839) + +- `redis.asyncio.cluster.RedisCluster:__init__` (full file reference) + +- `redis.asyncio.cluster.RedisCluster:execute_command` (full file reference) + +- `redis.asyncio.cluster.RedisCluster:pipeline` (full file reference) + + + + + +### Cluster Node Manager + +The Cluster Node Manager (`NodesManager`) is responsible for discovering and maintaining the topology of the Redis Cluster. It manages the cache of active nodes and their assigned slots, handles updates for MOVED exceptions, and initializes Redis connections to cluster nodes. It also integrates with a load balancer for read operations. + + + + + +**Related Classes/Methods**: + + + +- `redis.cluster.NodesManager:__init__` (full file reference) + +- `redis.cluster.NodesManager:initialize` (full file reference) + +- `redis.cluster.NodesManager:get_node_from_slot` (full file reference) + +- `redis.asyncio.cluster.NodesManager:__init__` (full file reference) + +- `redis.asyncio.cluster.NodesManager:initialize` (full file reference) + +- `redis.asyncio.cluster.NodesManager:get_node_from_slot` (full file reference) + + + + + +### Cluster Node Representation + +The Cluster Node Representation (`ClusterNode`) encapsulates the details of a single Redis instance within the cluster, including its host, port, name, and server type (primary or replica). It also holds a reference to the Redis connection object for that specific node. + + + + + +**Related Classes/Methods**: + + + +- `redis.cluster.ClusterNode:__init__` (full file reference) + +- `redis.asyncio.cluster.ClusterNode:__init__` (full file reference) + + + + + +### Load Balancer + +The Load Balancer (`LoadBalancer`) provides strategies for distributing read requests across multiple replica nodes associated with a primary. It supports round-robin and random replica selection to optimize read performance and distribute load. + + + + + +**Related Classes/Methods**: + + + +- `redis.cluster.LoadBalancer:get_server_index` (full file reference) + + + + + +### Cluster Pub/Sub + +The Cluster Pub/Sub (`ClusterPubSub`) component extends the standard Redis Pub/Sub functionality to work within a Redis Cluster environment. It manages the pub/sub connection to a specific cluster node, determined by channel keyslots or a random node. + + + + + +**Related Classes/Methods**: + + + +- `redis.cluster.ClusterPubSub:__init__` (full file reference) + +- `redis.cluster.ClusterPubSub:execute_command` (full file reference) + +- `redis.asyncio.cluster.ClusterPubSub:__init__` (full file reference) + + + + + +### Cluster Pipeline + +The Cluster Pipeline (`ClusterPipeline`) enables batching of multiple Redis commands for efficient execution within a Redis Cluster. It collects commands and then delegates their execution to specific strategies (PipelineStrategy or TransactionStrategy) based on whether a transaction is involved. + + + + + +**Related Classes/Methods**: + + + +- `redis.cluster.ClusterPipeline:__init__` (full file reference) + +- `redis.cluster.ClusterPipeline:execute` (full file reference) + +- `redis.asyncio.cluster.ClusterPipeline:__init__` (full file reference) + +- `redis.asyncio.cluster.ClusterPipeline:execute` (full file reference) + + + + + +### Pipeline Strategy + +The Pipeline Strategy (`PipelineStrategy`) defines how a batch of commands collected by the Cluster Pipeline is executed across the Redis Cluster. It handles routing commands to the correct nodes and processing their responses, without transactional guarantees. + + + + + +**Related Classes/Methods**: + + + +- `redis.cluster.PipelineStrategy:__init__` (full file reference) + +- `redis.cluster.PipelineStrategy:execute` (full file reference) + +- `redis.asyncio.cluster.PipelineStrategy:__init__` (full file reference) + +- `redis.asyncio.cluster.PipelineStrategy:execute` (full file reference) + + + + + +### Transaction Strategy + +The Transaction Strategy (`TransactionStrategy`) implements the logic for executing Redis transactions (MULTI/EXEC) within a Redis Cluster. It ensures that all commands within a transaction are sent to the same node and handles WATCH errors and retries. + + + + + +**Related Classes/Methods**: + + + +- `redis.cluster.TransactionStrategy:__init__` (full file reference) + +- `redis.cluster.TransactionStrategy:execute` (full file reference) + +- `redis.asyncio.cluster.TransactionStrategy:__init__` (full file reference) + +- `redis.asyncio.cluster.TransactionStrategy:execute` (full file reference) + + + + + +### Redis Sentinel Client + +The Redis Sentinel Client (`Sentinel`) provides an interface for applications to connect to Redis instances managed by Sentinel for high availability. It discovers the current master and available replicas for a given service name. + + + + + +**Related Classes/Methods**: + + + +- `redis.sentinel.Sentinel` (198:410) + +- `redis.sentinel.Sentinel:__init__` (227:249) + +- `redis.sentinel.Sentinel:master_for` (343:379) + +- `redis.asyncio.sentinel.Sentinel:__init__` (196:218) + +- `redis.asyncio.sentinel.Sentinel:master_for` (320:357) + + + + + +### Sentinel Connection Pool + +The Sentinel Connection Pool (`SentinelConnectionPool` and `SentinelManagedConnection`) manages the underlying network connections to Redis master and replica instances discovered via Sentinel. It ensures efficient connection reuse and handles connection lifecycle. + + + + + +**Related Classes/Methods**: + + + +- `redis.sentinel.SentinelConnectionPool:__init__` (145:166) + +- `redis.sentinel.SentinelManagedConnection:__init__` (21:23) + +- `redis.asyncio.sentinel.SentinelConnectionPool:__init__` (104:120) + +- `redis.asyncio.sentinel.SentinelManagedConnection:__init__` (27:29) + + + + + + + + + +### [FAQ](https://github.com/CodeBoarding/GeneratedOnBoardings/tree/main?tab=readme-ov-file#faq) \ No newline at end of file diff --git a/.codeboarding/Protocol_Command_Handling.md b/.codeboarding/Protocol_Command_Handling.md new file mode 100644 index 0000000000..fc2edd1376 --- /dev/null +++ b/.codeboarding/Protocol_Command_Handling.md @@ -0,0 +1,163 @@ +```mermaid + +graph LR + + Protocol_Serialization_Deserialization["Protocol Serialization & Deserialization"] + + Redis_Command_Abstraction_Implementation["Redis Command Abstraction & Implementation"] + + Core_Client_API["Core Client API"] + + Connection_Management["Connection Management"] + + Utility_Error_Handling["Utility & Error Handling"] + + Redis_Command_Abstraction_Implementation -- "uses" --> Protocol_Serialization_Deserialization + + Core_Client_API -- "uses" --> Redis_Command_Abstraction_Implementation + + Connection_Management -- "uses" --> Protocol_Serialization_Deserialization + + Protocol_Serialization_Deserialization -- "uses" --> Utility_Error_Handling + + Redis_Command_Abstraction_Implementation -- "uses" --> Utility_Error_Handling + + Core_Client_API -- "uses" --> Utility_Error_Handling + + Connection_Management -- "uses" --> Utility_Error_Handling + + click Connection_Management href "https://github.com/redis/redis-py/blob/master/.codeboarding//Connection_Management.md" "Details" + +``` + + + +[![CodeBoarding](https://img.shields.io/badge/Generated%20by-CodeBoarding-9cf?style=flat-square)](https://github.com/CodeBoarding/GeneratedOnBoardings)[![Demo](https://img.shields.io/badge/Try%20our-Demo-blue?style=flat-square)](https://www.codeboarding.org/demo)[![Contact](https://img.shields.io/badge/Contact%20us%20-%20contact@codeboarding.org-lightgrey?style=flat-square)](mailto:contact@codeboarding.org) + + + +## Details + + + +The `Protocol & Command Handling` subsystem is a critical part of the `redis-py` client library, responsible for the fundamental communication with the Redis server. + + + +### Protocol Serialization & Deserialization + +This component is responsible for the low-level conversion of Python data structures into the Redis Serialization Protocol (RESP) format for outgoing commands and the efficient parsing of RESP responses received from the Redis server back into native Python data types. It supports different RESP versions (RESP2 and RESP3) and leverages optimized parsers, including the `hiredis` C extension for performance. This component acts as the crucial bridge between the Python application's data representation and Redis's wire protocol. + + + + + +**Related Classes/Methods**: + + + +- `BaseParser` + +- `AsyncBaseParser` + +- `_RESPBase` + +- `_AsyncRESPBase` + +- `_HiredisParser` + +- `_AsyncHiredisParser` + +- `_RESP2Parser` + +- `_AsyncRESP2Parser` + +- `_RESP3Parser` + +- `_AsyncRESP3Parser` + +- `Encoder` + + + + + +### Redis Command Abstraction & Implementation + +This component defines and implements the comprehensive set of Redis commands, providing a Pythonic interface for interacting with Redis functionalities. It encompasses both the core Redis commands (e.g., key-value operations, hashes, lists, sets, sorted sets, streams, geospatial commands, hyperloglog, scanning, ACL, cluster, functions, management, pub/sub, scripting) and specialized commands for various Redis modules (e.g., RedisBloom, RedisJSON, RediSearch, RedisTimeSeries, VectorSearch). This component abstracts the raw command strings and arguments into user-friendly Python methods. + + + + + +**Related Classes/Methods**: + + + +- `CoreCommands` + +- `AsyncCoreCommands` + +- `RedisModuleCommands` + +- `AsyncRedisModuleCommands` + +- `BFCommands` + +- `CFCommands` + +- `JSONCommands` + +- `SearchCommands` + +- `AsyncSearchCommands` + +- `TimeSeriesCommands` + +- `VectorSetCommands` + + + + + +### Core Client API + +Provides a simplified, unified interface to the extensive set of Redis commands. + + + + + +**Related Classes/Methods**: _None_ + + + +### Connection Management [[Expand]](./Connection_Management.md) + +Responsible for the actual sending and receiving of bytes over the network. + + + + + +**Related Classes/Methods**: _None_ + + + +### Utility & Error Handling + +Provides consistent exception handling and potentially shared utility functions. + + + + + +**Related Classes/Methods**: _None_ + + + + + + + +### [FAQ](https://github.com/CodeBoarding/GeneratedOnBoardings/tree/main?tab=readme-ov-file#faq) \ No newline at end of file diff --git a/.codeboarding/Redis Client Core.md b/.codeboarding/Redis Client Core.md new file mode 100644 index 0000000000..3e4dabc7ff --- /dev/null +++ b/.codeboarding/Redis Client Core.md @@ -0,0 +1,239 @@ +```mermaid + +graph LR + + Redis_Client_Core["Redis Client Core"] + + Lock_Management["Lock Management"] + + Pub_Sub_Component["Pub/Sub Component"] + + Pipeline_Transaction_Component["Pipeline/Transaction Component"] + + Redis_Client_Core -- "creates instances of" --> Pub_Sub_Component + + Redis_Client_Core -- "creates instances of" --> Pipeline_Transaction_Component + + Lock_Management -- "utilizes" --> Redis_Client_Core + + Pipeline_Transaction_Component -- "extends functionality of" --> Redis_Client_Core + +``` + +[![CodeBoarding](https://img.shields.io/badge/Generated%20by-CodeBoarding-9cf?style=flat-square)](https://github.com/CodeBoarding/GeneratedOnBoardings)[![Demo](https://img.shields.io/badge/Try%20our-Demo-blue?style=flat-square)](https://www.codeboarding.org/demo)[![Contact](https://img.shields.io/badge/Contact%20us%20-%20contact@codeboarding.org-lightgrey?style=flat-square)](mailto:contact@codeboarding.org) + + + +## Component Details + + + +This graph illustrates the core components of the Redis client library, focusing on the primary interface for Redis interaction, `Redis Client Core`, and its specialized functionalities. It details how `Redis Client Core` orchestrates operations, including the creation of `Pub/Sub Component` for real-time messaging and `Pipeline/Transaction Component` for efficient batched commands. Additionally, it shows how `Lock Management` leverages the `Redis Client Core` for distributed locking, and how the `Pipeline/Transaction Component` builds upon the core client's capabilities. + + + +### Redis Client Core + +The Redis Client Core component serves as the primary interface for interacting with a Redis server. It manages connection pools, executes commands, handles command parsing, and provides entry points for advanced functionalities like pipelining and publish/subscribe. + + + + + +**Related Classes/Methods**: + + + +- `redis.client.Redis` (112:670) + +- `redis.client.Redis:from_url` (128:176) + +- `redis.client.Redis:from_pool` (179:192) + +- `redis.client.Redis:__init__` (199:389) + +- `redis.client.Redis:get_retry` (405:406) + +- `redis.client.Redis:set_retry` (408:410) + +- `redis.client.Redis:pipeline` (439:449) + +- `redis.client.Redis:transaction` (451:473) + +- `redis.client.Redis:pubsub` (556:564) + +- `redis.client.Redis:monitor` (566:567) + +- `redis.client.Redis:client` (569:572) + +- `redis.client.Redis:__exit__` (577:578) + +- `redis.client.Redis:__del__` (580:584) + +- `redis.client.Redis:_send_command_parse_response` (601:606) + +- `redis.client.Redis:execute_command` (622:623) + +- `redis.client.Redis:_execute_command` (625:644) + + + + + +### Lock Management + +The Lock Management component provides functionalities for acquiring, releasing, extending, and reacquiring distributed locks in Redis. It handles the underlying Redis commands and manages lock ownership and expiration, raising specific exceptions for lock-related errors. + + + + + +**Related Classes/Methods**: + + + +- `redis.lock.Lock` (14:343) + +- `redis.lock.Lock:__init__` (79:155) + +- `redis.lock.Lock:__enter__` (167:173) + +- `redis.lock.Lock:__exit__` (175:188) + +- `redis.lock.Lock:acquire` (190:235) + +- `redis.lock.Lock:release` (265:276) + +- `redis.lock.Lock:do_acquire` (237:245) + +- `redis.lock.Lock:do_release` (278:285) + +- `redis.lock.Lock:extend` (287:302) + +- `redis.lock.Lock:do_extend` (304:317) + +- `redis.lock.Lock:reacquire` (319:330) + +- `redis.lock.Lock:do_reacquire` (332:343) + + + + + +### Pub/Sub Component + +The Pub/Sub Component facilitates real-time messaging through Redis's publish/subscribe mechanism. It allows clients to subscribe to channels or patterns, receive messages, and manage the lifecycle of the pub/sub connection. + + + + + +**Related Classes/Methods**: + + + +- `redis.client.PubSub` (743:1000) + +- `redis.client.PubSub:__init__` (756:791) + +- `redis.client.PubSub:__exit__` (796:797) + +- `redis.client.PubSub:__del__` (799:806) + +- `redis.client.PubSub:close` (823:824) + +- `redis.client.PubSub:on_connect` (826:849) + +- `redis.client.PubSub:execute_command` (856:880) + +- `redis.client.PubSub:clean_health_check_responses` (882:898) + +- `redis.client.PubSub:_execute` (910:921) + +- `redis.client.PubSub:parse_response` (923:948) + +- `redis.client.PubSub:psubscribe` (983:1000) + +- `redis.client.PubSub:punsubscribe` (full file reference) + +- `redis.client.PubSub:subscribe` (full file reference) + +- `redis.client.PubSub:unsubscribe` (full file reference) + +- `redis.client.PubSub:ssubscribe` (full file reference) + +- `redis.client.PubSub:sunsubscribe` (full file reference) + +- `redis.client.PubSub:listen` (full file reference) + +- `redis.client.PubSub:get_message` (full file reference) + +- `redis.client.PubSub:ping` (full file reference) + +- `redis.client.PubSub:handle_message` (full file reference) + +- `redis.client.PubSub:run_in_thread` (full file reference) + + + + + +### Pipeline/Transaction Component + +The Pipeline/Transaction Component enables efficient execution of multiple Redis commands by sending them in a single round trip (pipelining) and supports atomic transactions using MULTI/EXEC. It manages command queuing, response parsing, and error handling for batched operations. + + + + + +**Related Classes/Methods**: + + + +- `redis.client.Pipeline` (1:1000) + +- `redis.client.Pipeline:__exit__` (full file reference) + +- `redis.client.Pipeline:__del__` (full file reference) + +- `redis.client.Pipeline:close` (full file reference) + +- `redis.client.Pipeline:multi` (full file reference) + +- `redis.client.Pipeline:execute_command` (full file reference) + +- `redis.client.Pipeline:_disconnect_reset_raise_on_watching` (full file reference) + +- `redis.client.Pipeline:immediate_execute_command` (full file reference) + +- `redis.client.Pipeline:_execute_transaction` (full file reference) + +- `redis.client.Pipeline:_execute_pipeline` (full file reference) + +- `redis.client.Pipeline:raise_first_error` (full file reference) + +- `redis.client.Pipeline:annotate_exception` (full file reference) + +- `redis.client.Pipeline:parse_response` (full file reference) + +- `redis.client.Pipeline:load_scripts` (full file reference) + +- `redis.client.Pipeline:_disconnect_raise_on_watching` (full file reference) + +- `redis.client.Pipeline:execute` (full file reference) + +- `redis.client.Pipeline:discard` (full file reference) + +- `redis.client.Pipeline:watch` (full file reference) + +- `redis.client.Pipeline:unwatch` (full file reference) + + + + + + + + + +### [FAQ](https://github.com/CodeBoarding/GeneratedOnBoardings/tree/main?tab=readme-ov-file#faq) \ No newline at end of file diff --git a/.codeboarding/Utility_Cross_Cutting_Concerns.md b/.codeboarding/Utility_Cross_Cutting_Concerns.md new file mode 100644 index 0000000000..b8e5e314b3 --- /dev/null +++ b/.codeboarding/Utility_Cross_Cutting_Concerns.md @@ -0,0 +1,201 @@ +```mermaid + +graph LR + + Authentication_Security["Authentication & Security"] + + Error_Handling["Error Handling"] + + Retry_Backoff_Strategies["Retry & Backoff Strategies"] + + Event_System["Event System"] + + Caching["Caching"] + + Distributed_Locks["Distributed Locks"] + + General_Utilities["General Utilities"] + + Authentication_Security -- "uses" --> Error_Handling + + Authentication_Security -- "dispatches events to" --> Event_System + + Retry_Backoff_Strategies -- "uses" --> Error_Handling + + Event_System -- "uses" --> Error_Handling + + Caching -- "uses" --> Error_Handling + + Distributed_Locks -- "uses" --> Error_Handling + + General_Utilities -- "uses" --> Error_Handling + + Authentication_Security -- "uses" --> General_Utilities + + Error_Handling -- "uses" --> General_Utilities + + Retry_Backoff_Strategies -- "uses" --> General_Utilities + + Event_System -- "uses" --> General_Utilities + + Caching -- "uses" --> General_Utilities + + Distributed_Locks -- "uses" --> General_Utilities + +``` + + + +[![CodeBoarding](https://img.shields.io/badge/Generated%20by-CodeBoarding-9cf?style=flat-square)](https://github.com/CodeBoarding/GeneratedOnBoardings)[![Demo](https://img.shields.io/badge/Try%20our-Demo-blue?style=flat-square)](https://www.codeboarding.org/demo)[![Contact](https://img.shields.io/badge/Contact%20us%20-%20contact@codeboarding.org-lightgrey?style=flat-square)](mailto:contact@codeboarding.org) + + + +## Details + + + +One paragraph explaining the functionality which is represented by this graph. What the main flow is and what is its purpose. + + + +### Authentication & Security + +This component is fundamental as it ensures secure communication and proper authorization, which are paramount for any client library interacting with a data store like Redis. It handles credential provision, token management, and certificate validation (e.g., OCSP stapling), directly addressing the "authentication mechanisms" aspect. + + + + + +**Related Classes/Methods**: + + + +- `redis/credentials.py` + +- `redis/auth/token.py` + +- `redis/auth/token_manager.py` + +- `redis/ocsp.py` + + + + + +### Error Handling + +This component is crucial for building a robust and user-friendly client library. A well-defined and comprehensive exception hierarchy allows developers to gracefully handle various issues, improving application stability and debugging. It directly addresses the "robust error handling" aspect. + + + + + +**Related Classes/Methods**: + + + +- `redis/exceptions.py` + + + + + +### Retry & Backoff Strategies + +This component is essential for resilience in distributed systems. Transient failures (e.g., network glitches, temporary server unavailability) are common, and automatic retries with intelligent backoff mechanisms prevent applications from crashing, reduce load on the server during recovery, and improve overall reliability. It directly addresses the "retry strategies" and "backoff" aspects. + + + + + +**Related Classes/Methods**: + + + +- `redis/retry.py` + +- `redis/backoff.py` + + + + + +### Event System + +This component provides a flexible and decoupled way for different parts of the library to communicate and react to internal state changes, such as connection status updates or re-authentication requirements. This promotes a clean architecture and enables dynamic behavior within the client. It directly addresses the "internal event system" aspect. + + + + + +**Related Classes/Methods**: + + + +- `redis/event.py` + + + + + +### Caching + +This component is fundamental for performance optimization in client libraries. By providing client-side caching capabilities, it reduces redundant network calls to the Redis server, significantly improving response times and reducing server load for frequently accessed data. It directly addresses the "caching" aspect. + + + + + +**Related Classes/Methods**: + + + +- `redis/cache.py` + + + + + +### Distributed Locks + +This component implements higher-level Redis commands that provide distributed synchronization mechanisms, most notably distributed locks. These primitives are essential for coordinating operations across multiple clients or processes in a distributed environment, preventing race conditions and ensuring data consistency. It directly addresses the "distributed synchronization primitives like locks" aspect. + + + + + +**Related Classes/Methods**: + + + +- `redis/lock.py` + +- `redis/asyncio/lock.py` + + + + + +### General Utilities + +This component provides a collection of general-purpose helper functions and utilities that are widely used across various parts of the library. It centralizes reusable code, preventing duplication, promoting consistency, and simplifying development across different modules. It directly addresses the "common utilities" aspect. + + + + + +**Related Classes/Methods**: + + + +- `redis/utils.py` + + + + + + + + + +### [FAQ](https://github.com/CodeBoarding/GeneratedOnBoardings/tree/main?tab=readme-ov-file#faq) \ No newline at end of file diff --git a/.codeboarding/on_boarding.md b/.codeboarding/on_boarding.md new file mode 100644 index 0000000000..58f9912307 --- /dev/null +++ b/.codeboarding/on_boarding.md @@ -0,0 +1,183 @@ +```mermaid + +graph LR + + Client_API["Client API"] + + Connection_Management["Connection Management"] + + Protocol_Command_Handling["Protocol & Command Handling"] + + Utility_Cross_Cutting_Concerns["Utility & Cross-Cutting Concerns"] + + Client_API -- "interacts with" --> Connection_Management + + Client_API -- "uses" --> Protocol_Command_Handling + + Connection_Management -- "provides connections to" --> Client_API + + Connection_Management -- "leverages" --> Utility_Cross_Cutting_Concerns + + Protocol_Command_Handling -- "used by" --> Client_API + + Protocol_Command_Handling -- "used by" --> Connection_Management + + Utility_Cross_Cutting_Concerns -- "supports" --> Client_API + + Utility_Cross_Cutting_Concerns -- "supports" --> Connection_Management + + click Client_API href "https://github.com/redis/redis-py/blob/master/.codeboarding//Client_API.md" "Details" + + click Connection_Management href "https://github.com/redis/redis-py/blob/master/.codeboarding//Connection_Management.md" "Details" + + click Protocol_Command_Handling href "https://github.com/redis/redis-py/blob/master/.codeboarding//Protocol_Command_Handling.md" "Details" + + click Utility_Cross_Cutting_Concerns href "https://github.com/redis/redis-py/blob/master/.codeboarding//Utility_Cross_Cutting_Concerns.md" "Details" + +``` + + + +[![CodeBoarding](https://img.shields.io/badge/Generated%20by-CodeBoarding-9cf?style=flat-square)](https://github.com/CodeBoarding/GeneratedOnBoardings)[![Demo](https://img.shields.io/badge/Try%20our-Demo-blue?style=flat-square)](https://www.codeboarding.org/demo)[![Contact](https://img.shields.io/badge/Contact%20us%20-%20contact@codeboarding.org-lightgrey?style=flat-square)](mailto:contact@codeboarding.org) + + + +## Details + + + +The redis-py library is designed with a clear layered architecture, where the Client API acts as the primary facade for users. This API delegates connection management to the Connection Management layer, which in turn relies on the Protocol & Command Handling layer for data serialization and deserialization. Various Utility & Cross-Cutting Concerns provide essential supporting services across these layers, ensuring robustness, security, and efficiency. + + + +### Client API [[Expand]](./Client_API.md) + +The primary interface for users to interact with Redis. It provides high-level methods for executing Redis commands, managing command pipelines, and handling transactions. It supports both synchronous and asynchronous operations, abstracting the underlying network communication. + + + + + +**Related Classes/Methods**: + + + +- `redis.client` (1:1) + +- `redis.asyncio.client` (1:1) + + + + + +### Connection Management [[Expand]](./Connection_Management.md) + +Responsible for establishing, maintaining, and pooling connections to various Redis deployments (standalone, Sentinel, Cluster). It handles connection lifecycle, including retries, error handling, and routing commands to appropriate nodes based on the deployment type. + + + + + +**Related Classes/Methods**: + + + +- `redis.connection` (1:1) + +- `redis.asyncio.connection` (1:1) + +- `redis.sentinel` (1:1) + +- `redis.asyncio.sentinel` (1:1) + +- `redis.cluster` (1:1) + +- `redis.asyncio.cluster` (1:1) + + + + + +### Protocol & Command Handling [[Expand]](./Protocol_Command_Handling.md) + +Encapsulates the logic for converting Python data into the Redis Serialization Protocol (RESP) format for outgoing commands and parsing RESP responses back into Python data types. It also defines the structure and arguments for all Redis commands, including specialized module commands (e.g., RedisJSON, RediSearch). + + + + + +**Related Classes/Methods**: + + + +- `redis._parsers.base` (1:1) + +- `redis._parsers.hiredis` (1:1) + +- `redis._parsers.resp2` (1:1) + +- `redis._parsers.resp3` (1:1) + +- `redis._parsers.encoders` (1:1) + +- `redis.commands.core` (1:1) + +- `redis.commands.bf` (1:1) + +- `redis.commands.json` (1:1) + +- `redis.commands.search` (1:1) + +- `redis.commands.timeseries` (1:1) + +- `redis.commands.vectorset` (1:1) + + + + + +### Utility & Cross-Cutting Concerns [[Expand]](./Utility_Cross_Cutting_Concerns.md) + +A collection of foundational services and common utilities that support the core functionality of the library. This includes authentication mechanisms, robust error handling, retry strategies, an internal event system, caching, and distributed synchronization primitives like locks. + + + + + +**Related Classes/Methods**: + + + +- `redis.credentials` (1:1) + +- `redis.auth.token` (1:1) + +- `redis.auth.token_manager` (1:1) + +- `redis.ocsp` (1:1) + +- `redis.exceptions` (1:1) + +- `redis.retry` (1:1) + +- `redis.backoff` (1:1) + +- `redis.event` (1:1) + +- `redis.cache` (1:1) + +- `redis.lock` (1:1) + +- `redis.asyncio.lock` (1:1) + +- `redis.utils` (1:1) + + + + + + + + + +### [FAQ](https://github.com/CodeBoarding/GeneratedOnBoardings/tree/main?tab=readme-ov-file#faq) \ No newline at end of file