Skip to content

Navigation Menu

Appearance settings

View all features
- BY COMPANY SIZE
  Enterprises
  Small and medium teams
  Startups
  Nonprofits
- BY USE CASE
  App Modernization
  DevSecOps
  DevOps
  CI/CD
  View all use cases
- BY INDUSTRY
  Healthcare
  Financial services
  Manufacturing
  Government
  View all industries
View all solutions
- EXPLORE BY TOPIC
  AI
  Software Development
  DevOps
  Security
  View all topics
- EXPLORE BY TYPE
  Customer stories
  Events & webinars
  Ebooks & reports
  Business insights
  GitHub Skills
- SUPPORT & SERVICES
  Documentation
  Customer support
  Community forum
  Trust center
  Partners
- COMMUNITY
  GitHub SponsorsFund open source developers
- PROGRAMS
  Security Lab
  Maintainer Community
  Accelerator
  Archive Program
- REPOSITORIES
  Topics
  Trending
  Collections
- ENTERPRISE SOLUTIONS
  Enterprise platformAI-powered developer platform
- AVAILABLE ADD-ONS
  GitHub Advanced SecurityEnterprise-grade security features
  Copilot for BusinessEnterprise-grade AI features
  Premium SupportEnterprise-grade 24/7 support
Pricing

Search code, repositories, users, issues, pull requests...

Search

Clear

Search syntax tips

Provide feedback

We read every piece of feedback, and take your input very seriously.

Include my email address so I can be contacted

Saved searches

Use saved searches to filter your results more quickly

Name

Query

To see all available qualifiers, see our documentation.

Appearance settings

You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session.

Dismiss alert

huggingface / transformers Public

Notifications You must be signed in to change notification settings
Fork 31.7k
Star 155k

Code
Issues 1.1k
Pull requests 1.1k
Actions
Projects 1
Security
Insights

Additional navigation options

Code
Issues
Pull requests
Actions
Projects
Security
Insights

Add Granite Speech Support#36801

Merged

eustlb merged 117 commits intohuggingface:mainhuggingface/transformers:mainfrom

alex-jw-brooks:granite_speechalex-jw-brooks/transformers:granite_speechCopy head branch name to clipboard

Apr 11, 2025

Conversation Commits117 (117)Checks Files changed

Merged

Add Granite Speech Support#36801
eustlb merged 117 commits intohuggingface:mainfrom
alex-jw-brooks:granite_speech

Commits

Commits on Apr 11, 2025

First pass at speech granite
alex-jw-brooks
committed
Combine into one model file with causal lm outputs for forward
alex-jw-brooks
committed
Add loss calc
alex-jw-brooks
committed
Fix config loading
alex-jw-brooks
committed
Split new / old loading logic
alex-jw-brooks
committed
Use transformers integration for loading peft adapters
alex-jw-brooks
committed
Add generation wrapper for selective lora enablement
alex-jw-brooks
committed
Add note for qformer encoder automodel
alex-jw-brooks
committed
Guard torch/audio imports in feature extractor
alex-jw-brooks
committed
Handle granite speech autoclasses
alex-jw-brooks
committed
Handle optional deps in package structure for granite speech
alex-jw-brooks
committed
Add granite pretrained model def for init
alex-jw-brooks
committed
Add dummy objects for torch/torchaudio
alex-jw-brooks
committed
Add tests for granite speech processor
alex-jw-brooks
committed
Minor formatting fixes and refactoring
alex-jw-brooks
committed
Add options for falling back to config in forward
alex-jw-brooks
committed
Tentative model docstrings for granite speech
alex-jw-brooks
committed
Fix config type
alex-jw-brooks
committed
Remove legacy load
alex-jw-brooks
committed
Allow non-lora variants for granite speech
alex-jw-brooks
committed
Override weight tying for llm
alex-jw-brooks
committed
Use text config instead of llm config
alex-jw-brooks
committed
Add output embeddings getter to fix weight tying
alex-jw-brooks
committed
Fix relative imports
alex-jw-brooks
committed
computing the number of audio features, based on the raw audio sequence.

avihu111
authored and
alex-jw-brooks
committed
collating audio inputs, and keeping the original lengths.

avihu111
authored and
alex-jw-brooks
committed
asserted we have text. otherwise we can't specify the audio special token.

avihu111
authored and
alex-jw-brooks
committed
assering the number of audio-symbols/audios match correctly.

avihu111
authored and
alex-jw-brooks
committed
indentation bugfix + supporting different feature lengths when expanding audio.

avihu111
authored and
alex-jw-brooks
committed
redundant, done in _get_validated_text

avihu111
authored and
alex-jw-brooks
committed
adapting the tests:

avihu111
authored and
alex-jw-brooks
committed
Minor cleanup, remove unused import
alex-jw-brooks
committed
Add more tests for batch feature processing
alex-jw-brooks
committed
Allow setting offset in rel position embeddings
alex-jw-brooks
committed
Add config option for warning if peft is not installed w/ lora
alex-jw-brooks
committed
Port blip2 qformer code into granite speech
alex-jw-brooks
committed
Add sad test for numpy arr processing
alex-jw-brooks
committed
Allow numpy arrays / tuples in granite speech processor
alex-jw-brooks
committed
Fix config type for projector
alex-jw-brooks
committed
- pad instead of creating a zeros tensor, to keep the original dtype/device (support bfloat16)

avihu111
authored and
alex-jw-brooks
committed
merge Blip2QFormerConfig to GraniteSpeechProjectorConfig

avihu111
authored and
alex-jw-brooks
committed
prevent a crash when re-saving/loading the model (line 109)

avihu111
authored and
alex-jw-brooks
committed
consider additional edge cases during preprocessing.

avihu111
authored and
alex-jw-brooks
committed
consider additional edge cases during preprocessing.

avihu111
authored and
alex-jw-brooks
committed
add features mask for batched inference (bugfix)

avihu111
authored and
alex-jw-brooks
committed
Minor refactor, remove multiaudio processor tests
alex-jw-brooks
committed
Add set input/output embeddings for granite speech
alex-jw-brooks
committed
Fix feature dim check in processor test
alex-jw-brooks
committed
Pop input features in embed test for granite speech
alex-jw-brooks
committed
Small fixes for test edge cases
alex-jw-brooks
committed
Add small tests for granite speech model
alex-jw-brooks
committed
Fix data parallelism test
alex-jw-brooks
committed
Standardize model class names
alex-jw-brooks
committed
Fix check for copies
alex-jw-brooks
committed
Fix misaligned init check
alex-jw-brooks
committed
Skip granite speech in checkpoint check
alex-jw-brooks
committed
Use default for tie_word_embeddings in granite speech
alex-jw-brooks
committed
Fix non documentation granite speech repo issues
alex-jw-brooks
committed
Fix comments and docstring checks
alex-jw-brooks
committed
Add placeholder docs for granite speech
alex-jw-brooks
committed
Fix test naming collision
alex-jw-brooks
committed
Code formatting
alex-jw-brooks
committed
Rerun torch dummy obj regen
alex-jw-brooks
committed
Fix save pretrained for granite speech
alex-jw-brooks
committed
Import sorting
alex-jw-brooks
committed
Fix tests typo
alex-jw-brooks
committed
Remove offset hack
alex-jw-brooks
committed
Pass args through encoder config
alex-jw-brooks
committed
Remove unused prune heads from blip2
alex-jw-brooks
committed
removing einsum. replaced with explicit multiplication (relative positional encodings) and sdpa attention.

avihu111
authored and
alex-jw-brooks
committed
remove Sequential from ConformerFeedForward and ConformerConvModule. + fix for sdpa attention

avihu111
authored and
alex-jw-brooks
committed
remove GraniteSpeechConformerScale

avihu111
authored and
alex-jw-brooks
committed
rename to hidden_states

avihu111
authored and
alex-jw-brooks
committed
rename conformer layers to self.layers, remove the first linear from the list to keep the list homogenous.

avihu111
authored and
alex-jw-brooks
committed
move pre-norm to the attention/feedforward blocks (avoid complex module wrapping)

avihu111
authored and
alex-jw-brooks
committed
adding pre_norm into forward

avihu111
authored and
alex-jw-brooks
committed
feature extractor refactoring to resemble how it's done in phi4multimodal.

avihu111
authored and
alex-jw-brooks
committed
rename feature_extractor to audio_processor

avihu111
authored and
alex-jw-brooks
committed
bugfix: input_feature_mask fix to get the exact number tokens.

avihu111
authored and
alex-jw-brooks
committed
Fix pytest decorator in processor test
alex-jw-brooks
committed
Add (disabled) integration tests for granite speech
alex-jw-brooks
committed
Fix handling of optional feature masking
alex-jw-brooks
committed
Loosen validation in processing for vLLM compatability
alex-jw-brooks
committed
Formatting fixes
alex-jw-brooks
committed
Update init structure to mirror llama
alex-jw-brooks
committed
Make granite speech projector generic
alex-jw-brooks
committed
Update test config to reflect generic projector
alex-jw-brooks
committed
Formatting fixes
alex-jw-brooks
committed
Fix typos, add license
alex-jw-brooks
committed
Fix undefined var in input processing
alex-jw-brooks
committed
Cleanup and expose ctc encoder
alex-jw-brooks
committed
Add missing config docstrings
alex-jw-brooks
committed
Better var names, type hints, etc
alex-jw-brooks
committed
Set attn context size in init
alex-jw-brooks
committed
Add max pos emb to encoder config
alex-jw-brooks
committed
Cleanup feature extractor
alex-jw-brooks
committed
Add granite speech architecture details
alex-jw-brooks
committed
Remove granite speech qformer ref
alex-jw-brooks
committed
Add paper link, explicit calc for qkv
alex-jw-brooks
committed
Calculate padding directly in depthwise conv1d init
alex-jw-brooks
committed
Raise value error instead of asserting
alex-jw-brooks
committed
Reorder class defs (classes used at top)
alex-jw-brooks
committed
Precompute relpos distances
alex-jw-brooks
committed
Run formatting
alex-jw-brooks
committed
Pass attention distances through forward
alex-jw-brooks
committed
Apply suggestions from code review

alex-jw-brooks
and
eustlb
committed
Add todo for using common batch feature extraction
alex-jw-brooks
committed
Rename audios/features
alex-jw-brooks
committed
Ensure chat template may be provided to processor
alex-jw-brooks
committed
Move granite speech docs to audio models
alex-jw-brooks
committed
Add todos for input proc refactoring
alex-jw-brooks
committed
Fix import order
alex-jw-brooks
committed
Guard torch import
alex-jw-brooks
committed
Use relative imports
alex-jw-brooks
committed
Require torch backend for processor in granite speech
alex-jw-brooks
committed
Add backend guards in feature extractor
alex-jw-brooks
committed
Merge branch 'main' into granite_speech
eustlb
authored

Footer

© 2026 GitHub, Inc.

Footer navigation

Terms
Privacy
Security
Status
Community
Docs
Contact

You can’t perform that action at this time.