You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This PR adds structured output support using Pydantic models, allowing users to define output schemas that LLM responses will be parsed into. Closes#121.
Structured outputs provide more reliable results by:
Constraining LLM outputs to specific schemas
Enabling built-in validation through Pydantic
Improving developer experience with typed responses
This directly addresses #121 by implementing the requested structured output functionality.
How to test
Define a Pydantic model for your desired output
Call get_parsed_completion() with your model class
Verify the response is properly parsed and validated
Example test cases are provided in the test file.
Risks & considerations
Currently only implemented for OpenAI - could be extended to other providers
Large/complex schemas may impact performance
Default error messages could be more user-friendly
Additional information
The implementation maintains backward compatibility while adding the new functionality. The parsing happens after the standard completion response, so existing code won't be affected.
Thank you for the PR. I have a couple of questions. First of all, it seems like the structured completion is only implemented at the client level and not exposed to any of the higher level APIs. Is this intentional? Also, currently the project uses unittest, not pytest, so it is better to stick to it.
Thank you for your feedback. Regarding the structured completion implementation:
The functionality is intentionally implemented at the client level (OpenAI-specific) while being properly abstracted in the base interface (BaseTextCompletionMixin._get_parsed_completion). This allows for:
Provider-specific implementations (currently only OpenAI supports this natively)
Consistent interface across providers
Flexibility for future provider integrations
I've reverted the tests back to unittest format - apologies for initially using pytest. The test cases now properly validate both the base interface and OpenAI-specific implementation.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR adds structured output support using Pydantic models, allowing users to define output schemas that LLM responses will be parsed into. Closes #121.
Changes
get_parsed_completion()function inskllm/llm/gpt/clients/openai/completion.pythat:tests/test_structured_outputs.pycovering:Motivation
Structured outputs provide more reliable results by:
This directly addresses #121 by implementing the requested structured output functionality.
How to test
get_parsed_completion()with your model classExample test cases are provided in the test file.
Risks & considerations
Additional information
The implementation maintains backward compatibility while adding the new functionality. The parsing happens after the standard completion response, so existing code won't be affected.