-
Notifications
You must be signed in to change notification settings - Fork 238
Support pre-registered plug-in NvTensorRtRtx execution provider library #1889
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
|
@gaugarg-nv @ishwar-raut1 for review |
|
Adding @kunal-vaishnavi and @baijumeswani for review |
src/models/model.cpp
Outdated
| if (provider_options.name != "cuda" && provider_options.name != "rocm" && provider_options.name != "DML") { | ||
| // Check if this is NvTensorRtRtx with a pre-registered provider (already handled above) | ||
| bool nvtrt_already_handled = false; | ||
| if (provider_options.name == "NvTensorRtRtx") { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we generalize to something like this?
bool already_handled = std::any_of(GetOrtEnv().GetEpDevices().begin(),
GetOrtEnv().GetEpDevices().end(),
[](const auto& device_ptr) { return device_ptr->Name() == "NvTensorRTRTXExecutionProvider"; });
if (already_handled) return;
// For providers that go through the extensible AppendExecutionProvider APIThere was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
|
@kunal-vaishnavi : I have addressed the review comments in the latest commit |
This PR basically extend the support for pre-registered plug-in for NvTenosrRtRTX which was earlier done for Cuda in the PR: #1850
What We Are Doing
Implementation Flow
Pre-Registered Plugin Path (PATH A)
When a custom EP library is registered
Built-in/Default Path (PATH B)
When no plugin is registered (standard behavior):