Use Case:

1. Login to DWP.
2. Search for some text like - Internal application OR IFSSC.
3. At top of the screen in browser - Toggle the button from Search/Chat.
4. This will initiate chatgpt.
5. The prompt shows message - I'm still working on getting you the information...

Result:
The results are displayed after 2.3 min.

Expected Result:
Result should be shown instantly.

Review assistant.log file located on Helix Assistant Pod.

Location: /opt/bmc/assistant/app/logs.

File Name: assistant.log 

log snippet:

-----------
Line 26839: 2024-12-16 11:57:17.697 | DEBUG    | 4250e854-57df-466f-a6ee-f76cbd0fb82c | 4d54feea425a4321a1ebdf3c11153d03 | <username> | openai._base_client:_request:1009 | HTTP Response: POST https://customer-env-helixgpt.openai.azure.com//openai/deployments/gpt-4/chat/completions?api-version=2024-06-01 "429 Too Many Requests" Headers({'content-length': '341', 'content-type': 'application/json', 'retry-after': '58', 'x-ratelimit-reset-tokens': '58', 'apim-request-id': '9df117df-a0e5-46c2-9336-a2244b44e2c0', 'strict-transport-security': 'max-age=31536000; includeSubDomains; preload', 'x-content-type-options': 'nosniff', 'policy-id': 'DeploymentRatelimit-Token', 'x-ms-region': 'East US 2', 'x-ratelimit-remaining-requests': '8', 'date': 'Mon, 16 Dec 2024 11:57:17 GMT'})
Line 26840: 2024-12-16 11:57:17.697 | DEBUG    | 4250e854-57df-466f-a6ee-f76cbd0fb82c | 4d54feea425a4321a1ebdf3c11153d03 | <username> | openai._base_client:_request:1017 | request_id: None
Line 26841: 2024-12-16 11:57:17.697 | DEBUG    | 4250e854-57df-466f-a6ee-f76cbd0fb82c | 4d54feea425a4321a1ebdf3c11153d03 | <username> | openai._base_client:_request:1022 | Encountered httpx.HTTPStatusError
Line 27007: 2024-12-16 11:57:17.701 | DEBUG    | 4250e854-57df-466f-a6ee-f76cbd0fb82c | 4d54feea425a4321a1ebdf3c11153d03 | <username> | openai._base_client:_should_retry:722 | Retrying due to status code 429
Line 27008: 2024-12-16 11:57:17.701 | DEBUG    | 4250e854-57df-466f-a6ee-f76cbd0fb82c | 4d54feea425a4321a1ebdf3c11153d03 | <username> | httpcore._trace:trace:45 | response_closed.started
Line 27009: 2024-12-16 11:57:17.701 | DEBUG    | 4250e854-57df-466f-a6ee-f76cbd0fb82c | 4d54feea425a4321a1ebdf3c11153d03 | <username> | httpcore._trace:trace:45 | response_closed.complete
Line 27010: 2024-12-16 11:57:17.701 | DEBUG    | 4250e854-57df-466f-a6ee-f76cbd0fb82c | 4d54feea425a4321a1ebdf3c11153d03 | <username> | openai._base_client:_retry_request:1064 | 1 retry left
Line 27011: 2024-12-16 11:57:17.701 | INFO     | 4250e854-57df-466f-a6ee-f76cbd0fb82c | 4d54feea425a4321a1ebdf3c11153d03 | <username> | openai._base_client:_retry_request:1069 | Retrying request to /chat/completions in 58.000000 seconds
-----------

Reason:

• 429 is a server side error regarding too many requests in a short period of time, so it is essentially the server saying that the client is being too chatty and to 'wait.
• When search execute, a post call is made to azure portal. Azure portal server returns error of HTTP 429. This indicates that the server has a configuration namely below that restricts the number of requests a client can make if that over exceeds the configured value.

Configuration Parameters to be adjusted so that the requests are not blocked by Azure portal.

 This is controlled by configuration
'requests per minute'
'Tokens per minute'

To fix this, above configurations need to be adjusted accordingly on Azure portal configurations. Can refer to the below link for more details:

https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/quota?tabs=rest