Skip to content

Reduce heap allocation during API transformations #563

@salmanap

Description

@salmanap

In #558 - we introduced the ability to transform and convert from one API format to another. This requires a lot of object transformations and in parts of the code we are over allocating the heap via TryFrom. We should use as little footprint as we can to reduce the memory allocation. Areas to improve

  1. In brightstaff, we need to look at the routing preferences and hence build two objects (one for the router and the other for the upstream llm call). See if we can converge to one
  2. The borrow and ownership semantics in request.rs and response.rs require a clone. Can we simplify that to ensure our object creation is simple and not over allocating heap?

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requesthelp wantedExtra attention is needed

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions