Content-Length: 352668 | pFad | http://github.com/NVIDIA/TensorRT-LLM/issues/#start-of-content

32 Issues · NVIDIA/TensorRT-LLM · GitHub
Skip to content

Issues: NVIDIA/TensorRT-LLM

Beta
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

596 Open 2,000 Closed
596 Open 2,000 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Assigned to nobody Loading
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

Feature request: pin versions of python packages in CI
#5002 opened Jun 7, 2025 by yuantailing
1
Cannot find 'setup.py' nor 'pyproject.toml' in TensorRT-LLM/3rdparty/cutlass/python bug Something isn't working
#4995 opened Jun 6, 2025 by hoangledoan
2 of 4 tasks
[Qwen2.5VL]: When can I use tensorrt_llm to deploy the qwen2.5vl model?
#4984 opened Jun 6, 2025 by HPUedCSLearner
Scaffolding tests failing on main branch with thread leaks and RuntimeError bug Something isn't working triaged Issue has been triaged by maintainers
#4974 opened Jun 6, 2025 by ccs96307
2
Feature Request: Enable chunked prefill by default in trtllm-serve or provide CLI flag feature request New feature or request. This includes new model, dtype, functionality support
#4947 opened Jun 5, 2025 by Nekofish-L
Feature Request: Add Llama_Nemotron_Nano_VL Support feature request New feature or request. This includes new model, dtype, functionality support
#4937 opened Jun 5, 2025 by guruprasad-atx
@laikhtewari
2
Feature Request: Add Prometheus Metrics Endpoint to trtllm-serve feature request New feature or request. This includes new model, dtype, functionality support
#4926 opened Jun 5, 2025 by Nekofish-L
[Nvidia A10G + _torch flow]: No fused attention + OOM for 2048 context length bug Something isn't working
#4917 opened Jun 4, 2025 by michaelfeil
4 tasks
1
CUDA error CUBLAS_STATUS_EXECUTION_FAILED when launching Qwen2.5-VL-72B using quickstart_multimodal.py bug Something isn't working
#4910 opened Jun 4, 2025 by CpyKing
2 of 4 tasks
[AutoDeploy] Expose logit_softcap in torch attention reference ops AutoDeploy
#4881 opened Jun 3, 2025 by lucaslie
@nvchenghaoz
[AutoDeploy] Expose logit_softcap parameter in flashinfer_attention AutoDeploy
#4880 opened Jun 3, 2025 by lucaslie
@nvchenghaoz
[AutoDeploy] Investigate DemoLLM Token Generation AutoDeploy bug Something isn't working
#4841 opened Jun 2, 2025 by lucaslie
@Fridah-nv
Title: KeyError: 'gemma3' error in GemmaConfig.from_hugging_face when converting Gemma 3 model bug Something isn't working triaged Issue has been triaged by maintainers
#4825 opened Jun 2, 2025 by bebilli
2 of 4 tasks
@brb-nv
3
Driver crash during warmup of DeepSeek-R1-FP4 bug Something isn't working
#4816 opened May 31, 2025 by pathorn
1 of 4 tasks
3
The output of Gemma 3 4B for TensorRT and Transformers is not the same, even when using float32 bug Something isn't working triaged Issue has been triaged by maintainers
#4815 opened May 31, 2025 by Alireza3242
1 of 4 tasks
@brb-nv
1
[Bug] Users need to add cuda_graph_max_batch_size=0 to avoid crash when config from extra-llm-api-config.yml bug Something isn't working
#4811 opened May 30, 2025 by chang-l
4 tasks
@Superjomn
2
Inconsistent output_log_probs with concurrent requests at beam_width and max_batch_size ≥ 2 bug Something isn't working
#4793 opened May 30, 2025 by wonjkim
4 tasks
Gemma-2 Style Attention Pattern Matching with logit softcap AutoDeploy
#4789 opened May 30, 2025 by lucaslie
@lucaslie @nvchenghaoz
1
llmapi usage: how to add callback after each step and embedding table in LLM.generate_async
#4788 opened May 30, 2025 by bnuzhanyu
Feature support: eagle multimodal inputs feature request New feature or request. This includes new model, dtype, functionality support
#4787 opened May 30, 2025 by liyi-xia
@laikhtewari
1
Patch for create_causal_mask() function in transformers masking_utils.py AutoDeploy
#4783 opened May 30, 2025 by sugunav14
1
Priority for stop_token_ids vs. min_tokens
#4776 opened May 29, 2025 by jkbeavers
Retouch ccp executor example cmake to enable or not multi device building
#4770 opened May 29, 2025 by WilliamTambellini
@tongyuantongyu
3
How is the performance of the model with pytorch as the backend Investigating Performance TRTLLM model inference speed, throughput, efficiency. Latency, benchmarks, regressions, opts. triaged Issue has been triaged by maintainers
#4745 opened May 29, 2025 by oppolll
@byshiue @hypdeb
2
Test gemma models after upgrade to latest transformers AutoDeploy
#4740 opened May 28, 2025 by sugunav14
@sugunav14
ProTip! Find all open issues with in progress development work with linked:pr.








ApplySandwichStrip

pFad - (p)hone/(F)rame/(a)nonymizer/(d)eclutterfier!      Saves Data!


--- a PPN by Garber Painting Akron. With Image Size Reduction included!

Fetched URL: http://github.com/NVIDIA/TensorRT-LLM/issues/#start-of-content

Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy