Low-Rank Adaptation of Large Multimodal Models forResource-Constrained Vision Applications

Authors

  • Brock Donovan Faculty of Engineering and Information Technology, University of Melbourne, Melbourne, Australia Author
  • Nanami Inoue Faculty of Engineering and Information Technology, University of Melbourne, Melbourne, Australia Author
  • Tsubasa Nakajima Faculty of Engineering and Information Technology, University of Melbourne, Melbourne, Australia Author

DOI:

https://doi.org/10.66525/ijictt91

Keywords:

Large Multimodal Models, Low-Rank Adaptation, Edge Computing, Computer Vision, Parameter-Efficient Fine-Tuning

Abstract

The integration of visual and textual modalities within unified
computational frameworks has led to the emergence of large multimodal models,
which exhibit unprecedented capabilities in tasks ranging from visual question
answering to complex scene understanding. However, the immense parameter
counts of these models present formidable barriers to their deployment in resourceconstrained environments, such as edge devices, mobile platforms, and embedded
vision systems. This paper provides a comprehensive investigation into the
application of low-rank adaptation techniques as a viable pathway for deploying
large multimodal models under strict hardware limitations. By systematically
decoupling the adaptation process from the full parameter space, low-rank
adaptation effectively reduces the memory and computational footprints required
for domain-specific fine-tuning. We explore the architectural modifications
necessary to implement these techniques across both the visual encoder and the
cross-modal projection layers without compromising the inherent generalization
capabilities of the foundational model. Furthermore, this study evaluates the tradeoffs between parameter efficiency, inference latency, and task performance in
realistic edge-computing scenarios. Through extensive analysis, we demonstrate
that optimal matrix decomposition strategies can achieve performance parity with
full fine-tuning while reducing trainable parameters by several orders of
magnitude. The findings presented herein offer critical insights for researchers and
practitioners seeking to democratize access to advanced visual intelligence in
power-limited and memory-restricted applications.

Downloads

Download data is not yet available.

References

Zhao, H., Gu, J., Wang, S., Lu, T., Zhang, X., Wu, Z., ... & Jiang, Y. G. (2026). LSTD: Long

Short-Term Temporal Diffusion for Video Generation. IEEE Transactions on

Multimedia.

Wang, C., Song, R., Muller, R., Monteuuis, J. P., Celik, Z. B., Petit, J., ... & Li, M. (2026,

March). Cp-freezer: Latency attacks against vehicular cooperative perception. In

Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 40, No. 2, pp.

1114-1122).

Peng, Q., Zheng, C., & Chen, C. (2024). A dual-augmentor framework for domain

generalization in 3d human pose estimation. In Proceedings of the IEEE/CVF

Conference on Computer Vision and Pattern Recognition (pp. 2240-2249).

Zhu, Guoying, et al. "Enabling MoE on the Edge via Importance-Driven Expert Scheduling."

arXiv preprint arXiv:2508.18983 (2025).

Huang, J., Tao, A., Marco, R., Bogdanovic, M., Kelly, J., & Shkurti, F. (2025, May).

Automated planning domain inference for task and motion planning. In 2025 IEEE

International Conference on Robotics and Automation (ICRA) (pp. 12534-12540).

IEEE.

Wang J, Fan L, Li B, et al. A Dynamic Factor Gating Architecture with Market Regime

Awareness for Stock Return Forecasting[J]. 2026.

Lv, Qi, et al. "F1: A vision-language-action model bridging understanding and generation to

actions." arXiv preprint arXiv:2509.06951 (2025).

Cai, Y., Jing, T., Liu, C., Lian, Z., Huang, S. S., & Huang, H. (2025, June). Spatio-Temporally

Consistent Depth Estimation for Dynamic Scenes using 3D Scene Flows. In 2025 IEEE

International Conference on Multimedia and Expo (ICME) (pp. 1-6). IEEE.

Tu, P., Xie, X., Ai, G., Li, Y., Huang, Y., & Zheng, Y. (2023). FemtoDet: An object detection

baseline for energy versus performance tradeoffs. In Proceedings of the IEEE/CVF

International Conference on Computer Vision (pp. 13318-13327).

Wang, S., Yu, Y., Feldt, R., & Parthasarathy, D. (2025). Automating a complete software test

process using llms: An automotive case study. arXiv preprint arXiv:2502.04008.

Dong, J., Liu, J., Qu, X., & Ong, Y. S. (2025). Confound from All Sides, Distill with Resilience:

Multi-Objective Adversarial Paths to Zero-Shot Robustness. In Proceedings of the

IEEE/CVF International Conference on Computer Vision (pp. 624-634).

Huang, H., Zhang, J., Yu, L., Zhang, J., Wu, Q., & Xu, C. (2021). TOAN: Target-oriented

alignment network for fine-grained image categorization with few labeled samples.

IEEE Transactions on Circuits and Systems for Video Technology, 32(2), 853-866.

Lv, Qi, et al. "Spatial-temporal graph diffusion policy with kinematic modeling for bimanual

robotic manipulation." Proceedings of the Computer Vision and Pattern Recognition

Conference. 2025.

Zhao, H., Lu, T., Gu, J., Zhang, X., Zheng, Q., Wu, Z., ... & Jiang, Y. G. (2024, September).

Magdiff: Multi-alignment diffusion for high-fidelity video generation and editing.

226

In European Conference on Computer Vision (pp. 205-221). Cham: Springer Nature

Switzerland.

Zhang, P., Zhu, S., Wang, C., Zhao, Y., & Lam, E. Y. (2024). Neuromorphic imaging with

super-resolution. IEEE Transactions on Circuits and Systems for Video Technology,

35(2), 1715-1727.

Tang, Y., Zhang, G., Liu, J. K., & Qin, R. (2025). Weakly supervised land-cover classification

of high-resolution images with low-resolution labels through optimized label

refinement. International Journal of Remote Sensing, 46(5), 1913-1937.

Tu, P., Huang, Y., Zheng, F., He, Z., Cao, L., & Shao, L. (2022, June). Guidedmix-net: Semisupervised semantic segmentation by using labeled images as reference. In Proceedings

of the AAAI conference on artificial intelligence (Vol. 36, No. 2, pp. 2379-2387).

Dong, J., Koniusz, P., Zhang, Y., Zhu, H., Liu, W., Qu, X., & Ong, Y. S. (2025, October).

Improving zero-shot adversarial robustness in vision-language models by closed-form

alignment of adversarial path simplices. In Forty-second International Conference on

Machine Learning.

Muller, R., Song, R., Wang, C., Zhan, Y., Monteuuis, J. P., Man, Y., ... & Celik, Z. B. (2025,

May). Investigating physical latency attacks against camera-based perception. In 2025

IEEE Symposium on Security and Privacy (SP) (pp. 4588-4605). IEEE.

Mi, L., Wang, W., Tu, W., He, Q., Kong, R., Fang, X., ... & Liu, Y. (2025, March). Empower

vision applications with LoRA LMM. In Proceedings of the Twentieth European

Conference on Computer Systems (pp. 261-277).

Dong, J., Wang, Y., Lai, J., & Xie, X. (2023). Restricted black-box adversarial attack against

deepfake face swapping. IEEE Transactions on Information Forensics and Security, 18,

2596-2608.

Peng, Q., Xue, H., Wang, P., & Chen, C. (2026, March). Lifelong Domain Adaptive 3D Human

Pose Estimation. In Proceedings of the AAAI Conference on Artificial Intelligence

(Vol. 40, No. 10, pp. 8358-8366).

Qu, W., Shao, Y., Meng, L., Huang, X., & Xiao, L. (2024). A conditional denoising diffusion

probabilistic model for point cloud upsampling. In Proceedings of the IEEE/CVF

Conference on Computer Vision and Pattern Recognition (pp. 20786-20795).

Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., et al. (2021). Learning

transferable visual models from natural language supervision. In Proceedings of ICML.

Qi, Z., Wang, S., Zhang, W., & Huang, Q. (2024). Uncertainty-boosted robust video activity

anticipation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(12),

7775-7792.

Liu, Y., & Kwon, H. (2025). Efficient Depth Estimation for Unstable Stereo Camera Systems

on AR Glasses. In Proceedings of the IEEE/CVF Conference on Computer Vision and

Pattern Recognition (pp. 6252-6261).

Zhang, S., Yang, S., Zhang, W., Xiong, Y., & Yao, S. (2026). Hybrid Beamforming for

Subarray-Level Movable Antenna Enhanced MU-MIMO Communications. IEEE

Wireless Communications Letters, 15, 2559-2563.

Dai, S., Wu, Y., Chen, S., Huang, R., & Dannenberg, R. B. (2023, November). SingStyle111:

A Multilingual Singing Dataset With Style Transfer. In ISMIR (pp. 765-773).

Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., et al.

(2021). An image is worth 16x16 words: Transformers for image recognition at scale.

In Proceedings of ICLR.

Xie, C., Zhu, D., Wang, Z., Zhang, H., & Wei, Z. (2026). Compliance-Aware Discharge Agent

for Auditable ICU Discharge Planning: A Pilot Feasibility Study Using Structured

eICU Records. Available at SSRN 6429758.

Xie, C., Zhu, D., Wang, Z., Zhang, H., & Wei, Z. (2026). Explainable AI for Mental Health

Detection from Social Media: A Comparative Study of Traditional Machine Learning

and a Large Language Model. Available at SSRN 6429778.

Downloads

Published

2026-05-10