Kourosh Meshgi

RESEARCH SCIENTIST @ RIKEN CENTER OF ADVANCED INTELLIGENCE PROJECT (AIP)

I am currently a research scientist for the Language Information Access Technology (LIAT) team , RIKEN AIP under the supervision of Prof. Satoshi Sekine. I am based in Seattle and have extensive knowledge in machine learning, particularly deep learning and generative AI, not only fine-tuning models but also designing and developing novel architectures and publishing them. I work on transferring my knowledge from CV to NLP to help knowledge extraction from text, constructing a bridging between large language models (LLMs) and CV tasks, structuring it for higher-level inferences using multi-task learning, as well as delving into the realm of transformers and employing them to solve problems quickly and reliably. Before that, I was a postdoctoral fellow at the Graduate School of Informatics at Kyoto University, where as a member of the Ishii lab, I worked with Prof. Shin Ishii and Dr. Shigeyuki Oba on the use of active learning in visual tracking and scene understanding, funded by the project of strategic advancement of multi-purpose ultra-human robot and artificial intelligence technologies (SAMURAI) from the New Energy and Industrial Technology Development Organization (NEDO) and Post-Kei project. I am also collaborating with Biochemical Simulation Lab from RIKEN QBiC, RIKEN AIP's Machine Intelligence for Medical Engineering Team, and Neural Computation Unitof OIST on the Exploratory Challenges of Post-K project. I also collaborate with Human-AI Communication team of RIKEN AIP under the supervision of Prof. Toyoaki Nishida.

I finished my Ph.D. under the supervision of Prof. Shin Ishii in the same lab, where I worked on occlusion-aware visual tracking, 3D object reconstruction, and imitation learning. In particular, my Ph.D. dissertation was about particle filter-based tracking to handle persistent and complex occlusions by proposing novel occlusion-aware appearance model and context-aware motion model in this framework.

Bio: Kourosh Meshgi received his B.Sc. and M.Sc. in Hardware Engineering (2008) and Artificial Intelligence (2010) respectively, from Tehran Polytechnic and his Ph.D. in Informatics in 2015 from Kyoto University. He is currently a research scientist at RIKEN AIP (remotely working from Seattle, WA) and a guest researcher at Kyoto University. His research interests include machine learning, natural language processing, computer vision, robotics, and computational linguistics. Dr. Meshgi's current research focuses on machine learning, generative AI, and designing, training, deploying, fine-tuning, interpreting, explaining, and orchestrating deep learning models on NLP and CV tasks.

Curriculum Vitae

Publications

Natural Language Processing

M. S. Mirzaei, K. Meshgi, and S. Sekine, “What is the real intention behind this question? Dataset collection and intention classification”, In Proc. Of ACL’23, Toronto, Canada, Jul 2023.
M. S. Mirzaei, K. Meshgi, and S. Sekine, “Is this Question Real? Dataset Collection on Perceived Intentions and Implicit Attack Detection”, In Proc. of WWW’22, Lyon, France, Apr 2022.
K. Meshgi, M. S. Mirzaei, and S. Sekine, “Uncertainty Regularized Multi-Task Learning,” In Proc. of ACL’22, WASSA workshop, Dublin, Ireland, May 2022.
K. Meshgi, M. S. Mirzaei, and S. Sekine, “Q-Learning Scheduler for Multi Task Learning Through the use of Histogram of Task Uncertainty,” In Proc. of ACL’22, RepL4NLP workshop, Dublin, Ireland, May 2022.
K. Meshgi, and M. S. Mirzaei, “Using Uncertainty for Multi-Domain Text Classification”, In ExLing’20, Athens, Greece, Oct 2020.
S. Ebrahimi, K. Meshgi, S. Khadivi, S.E. Shiri Ahmad Abady, “Meta-level Statistical Machine Translation," in Proc. of IJCNLP'13, Nagoya, Japan, Oct 2013.

Computer Vision

K. Meshgi, M. S. Mirzaei, S. Sekine, “Transformer-based Multi-Task Representation Learning for Robust Visual Tracking,” In Proc. of ICIP’24, IEEE, Abu Dhabi, UAE, Oct 2024 (submitted).
A. Kubo, K. Meshgi, S. Ishii, “Adaptive Correlation Filters for Visual Tracking via Reinforcement Learning,” J. of Intelligent & Robotic Systems, 2021.
K. Meshgi, M. S. Mirzaei, S. Oba, “Adversarial Semi-Supervised Multi-Domain Tracking,” In Proc. of ACCV’20, IEEE, Kyoto, Japan, Nov 2020.
K. Meshgi, M. S. Mirzaei, and S. Oba, “Leveraging Tacit Information Embedded in CNN Layers for Visual Tracking,” In Proc. of ACCV’20, IEEE, Kyoto, Japan, Nov 2020.
K. Meshgi, M. S. Mirzaei, and S. Sekine, “Uncertainty Regularized Multi-Task Learning for Text Classification,” in Proc. of ACML’19, Weakly Supervised Learning Workshop, Nagoya, Japan, Nov 2019.
K. Meshgi, M. S. Mirzaei, S. Oba, “Long and Short Memory Balancing in Visual Co-Tracking using Q-Learning,” in Proc. of ICIP’19, IEEE, Taipei, Taiwan, Sep 2019.
K. Meshgi, M. S. Mirzaei, S. Oba, “Information-Maximizing Sampling to Promote Tracking-by-Detection,” in Proc. of ICIP’18, IEEE, Athens, Greece, Oct 2018.
K. Meshgi, S. Oba, S. Ishii, “Efficient Diverse Ensemble for Discriminative Co-Tracking,” in Proc. of CVPR’18, IEEE, Salt Lake City, USA, Jun 2018.
K. Meshgi and S. Oba “Active Collaboration of Classifiers for Visual Tracking,” in G. Anbarjafari and S. Escalera (Eds) Human-Robot Interaction - Theory and Application, InTech Publication, ISBN 978-953-51-5611-6, 2018.
K. Meshgi, S.i. Maeda, S. Oba, S. Ishii, “Constructing a Meta-Tracker using Dropout to Imitate the Behavior of an Arbitrary Black-box Tracker,” Journal of Neural Networks vol. 87, pp. 132-148, Elsevier, 2017.
K. Meshgi, M. S. Mirzaei, S. Oba, S. Ishii, “Efficient Asymmetric Co-Tracking using Uncertainty Sampling,” in Proc. of ICSIPA’17, IEEE, Kuching, Malaysia, Sep 2017. (Best paper award)
K. Meshgi, M. S. Mirzaei, S. Oba, S. Ishii, “Active Collaborative Ensemble Tracking,” in Proc. of AVSS’17, IEEE, Lecce, Italy, Aug 2017.
K. Meshgi, S. Oba, S. Ishii, “Adversarial Sampling to Robustify Active Discriminative Co-Tracking,” in Proc. of MIRU’17, Hiroshima, Japan, Aug 2017.
K. Meshgi, S. Oba, S. Ishii, “Efficient Version-Space Reduction for Visual Tracking,” in Proc. of CRV’17, IEEE, Vancouver, Canada, May 2017.
K. Meshgi, S. Oba, S. Ishii, “Active Discriminative Tracking using Collective Memory,” in Proc. of MVA’17, IEEE, Tokyo, Japan, May 2017.
K. Meshgi, S. Oba, and S. Ishii, “Robust Discriminative Tracking via Query-by-Bagging,” in Proc. of AVSS’16, Colorado Springs, USA, Aug 2016.
K. Meshgi, S.i. Maeda, S. Oba, H. Skibbe, Y.Z. Li, S. Ishii, “Occlusion Aware Particle Filter Tracker to Handle Complex and Persistent Occlusions,” Journal of Computer Vision and Image Understanding (CVIU), vol. 150, pp. 81-94, Elsevier 2016.
K. Meshgi, S.i. Maeda, S. Oba, and S. Ishii, “Data-driven Probabilistic Occlusion Mask to Promote Visual Tracking,” in Proc. of CRV’16, IEEE, British Columbia, Canada, Jun 2016.
K. Meshgi, S. Ishii, “The State-of-the-Art in Handling Occlusions for Visual Object Tracking,” IEICE Transactions on Information and Systems, vol. E98-D, no. 7, pp. 1260-1274, IEICE 2015.
K. Meshgi, and S. Ishii, “Expanding Histogram of Colors with Gridding to Improve Tracking Accuracy,” in Proc. of MVA’15, IEEE, Tokyo, Japan, May 2015.
K. Meshgi, S. Maeda, S. Oba, S. Ishii, “Fusion of Multiple Cues from Color and Depth Domains using Occlusion Aware Bayesian Tracker, ” in IEICE Tech. Rep., vol. 113, no. 500, NC2014-22, pp. 127-132, Mar 2014.
K. Meshgi, Y.Z. Li, S. Oba, S. Maeda, S. Ishii, “Enhancing Probabilistic Appearance-Based Object Tracking with Depth Information: Object Tracking under Occlusion,” in IEICE Tech. Rep., vol. 113, no. 197, IBISML2013-22, pp. 85-91, Sep 2013.
K. Meshgi, “Particle Filter-based Tracking to Handle Persistent and Complex Occlusions and Imitate Arbitrary Black-box Trackers,” Ph.D. Dissertation, Kyoto University, Sep 2015.

Computational Linguistics

M. S. Mirzaei, and K. Meshgi, “The use of machine learning in developing learner-adaptive tools for second language acquisition”, In Proc. of EuroCALL’23, Reykjavík, Iceland, Aug 2023.
M. S. Mirzaei, and K. Meshgi, “Self-regulated listening experience with smart captioning”, In Proc. of EuroCALL’23, Reykjavík, Iceland, Aug 2022.
M. S. Mirzaei, and K. Meshgi, “Adaptive Listening Difficulty Detection for L2 Learners Through Moderating ASR Resources”, In Proc. of Interspeech’21, Brno, Czechia, Aug 2021.
M. S. Mirzaei, and K. Meshgi, “Facilitating L2 listening through automatic detection of speech and lexical difficulties”, In Proc. of EuroCALL’21, Paris, France, Aug 2021.
M. S. Mirzaei, K. Meshgi, and T. Nishida, “Promoting common ground building in L2 cross-cultural conversations”, In Proc. of EuroCALL’21, Paris, France, Aug 2021.
M. S. Mirzaei, K. Meshgi, and T. Nishida “Sentence Complexity as an Indicator of L2 Learner’s Listening Difficulty,” In Proc. of EuroCALL’20, Copenhagen, Denmark, Aug 2020.
M. S. Mirzaei, K. Meshgi, and T. Nishida, “A Situation Creation System to Enable Experiential Learning in Virtual Worlds for Developing Cross-Cultural Competencies,” In Proc. of EuroCALL’20, Copenhagen, Denmark, Aug 2020.
M. S. Mirzaei, and K. Meshgi, “A Data-driven Caption for L2 Listening”, In Proc. of ExLing’20, Athens, Greece, Oct 2020.
M. S. Mirzaei, K. Meshgi, “Learner Adaptive Partial and Synchronized Caption for L2 Listening Skill Development,” in Proc. of EuroCALL’19, Louvain-la-Neuve, Belgium, Aug 2019.
M. S. Mirzaei, K. Meshgi, “Toward Adaptive Partial and Synchronized Caption to Facilitate L2 Listening,” in Proc. of FLEAT VII, Tokyo, Japan, Aug 2019.
K. Meshgi, M. S. Mirzaei, “A comprehensive word difficulty index for L2 listening,” in Proc. of ExLing’18, Paris, France, Aug 2018.
K. Meshgi, M. S. Mirzaei, “A Data-driven Approach to Generate Partial and Synchronized Caption for Second Language Listeners,” in Proc. of WorldCALL’18, Concepcion, Chile, Nov 2018.
M. S. Mirzaei, K. Meshgi, “Automatic Scaffolding for L2 Listeners by Leveraging Natural Language Processing,” in Proc. of EuroCALL’18, Jyväskylä, Finland, Aug 2018.
M. S. Mirzaei, K. Meshgi, T. Kawahara, “Exploiting Automatic Speech Recognition Errors to Enhance Partial and Synchronized Caption for Facilitating Second Language Listening,” Computer Speech and Language Journal, vol. 49, pp. 17-36, Elsevier 2018.
M. S. Mirzaei, K. Meshgi, Y. Akita, T. Kawahara, “Partial and synchronized captioning: A new tool to assist learners in developing second language listening skill,” ReCALL Journal, vol. 29(2), pp. 178-199, Cambridge University Press 2017.
M. S. Mirzaei, K. Meshgi, T. Kawahara, “Detecting listening difficulty for second language learners using Automatic Speech Recognition errors,” in Proc. of SLaTE’17, Stockholm, Sweden, Aug 2017.
M. S. Mirzaei, K. Meshgi, T. Kawahara, “Listening Difficulty Detection to Foster Second Language Listening with Partial and Synchronized Caption,” in Proc. of EuroCALL’17, Southampton, England, Aug 2017.
M. S. Mirzaei, K. Meshgi, T. Kawahara, “Leveraging automatic speech recognition errors to detect challenging speech segments in TED talks,” in Proc. of EuroCALL’16, Limassol, Cyprus, Aug 2016.
M. S. Mirzaei, K. Meshgi, T. Kawahara, “Automatic speech recognition errors as a predictor of L2 listening difficulties,” in Proc. of Coling’16 (CL4LC Workshop), Osaka, Japan, Nov 2016.
M. S. Mirzaei, K. Meshgi, Y. Akita, T. Kawahara, “Errors in Automatic Speech Recognition versus Difficulties in Second Language Listening, ” in Proc. of EuroCALL’15, Padova, Italy, Aug 2015.

Machine Learning

M. S. Mirzaei, K. Meshgi, E. Frigo, and T. Nishida, “AnimGAN: A Spatiotemporally-Conditioned Generative Adversarial Network for Character Animation,” in Proc. of ICIP’20, IEEE, Abu Dhabi, UAE, Oct 2020.
S. MasoumZadeh, K. Meshgi, S. Shiry, G. Taghizadeh, “FQL-RED: An Adaptive Scalable Schema for Active Queue Management,” Int'l. J. of Network Mgmt (IJNM), vol. 21, pp.157-167 Wiley, 2011.
S. MasoumZadeh, G. Taghizadeh, K. Meshgi, S. Shiry, “Deep Blue: A Fuzzy Q-Learning Enhanced Active Queue Management Scheme,” in Proc. of Int'l. Conf. on Adaptive and Intelligent Systems (ICAIS'09), Klagenfurt, Austria, 2009.
S. MasoumZadeh, K. Meshgi, S. Shiry, “Adaptive Mutation in Evolution Strategy using Fuzzy Q-Learning,” in Proc. of 13th Iran Computer Association Conf. (ICCSC2008), Kish, Iran, 2008.
K. Meshgi, “Brain Inspired Face Detection,” M.Sc. Dissertation, Tehran PolyTechnic, Oct 2010.

Robotics

S. Soleimanpour, S. Shiry, K. Meshgi, “Sensor Fusion in Robot Localization using DS-Evidence Theory with Conflict Detection using Mahalanobis Distance,” Proc. of 7th IEEE Int'l. Conf. on Cybernetic Intelligent Systems (CIS'2008), United Kingdom, 2008.
Nemesis 2010 Team Description, RobuCup 2010.

Legend: , , ,

Activities

Honors

Awarded JSPS Kakenhi (Grants-in-Aid for Scientific Research), Japan Society for the Promotion of Science, 2017.
Won ICT Innovation Award for PSC, Kyoto University, 2017.
Awarded as Japan Ministry of Economy, Trade and Industry Prize for Winning NEDO Project, as a part of R&D Team, 3D MEDiA Co. Ltd., 2015.
Received Japan Government Monbokagakusho (MEXT) Scholarship from the Ministry of Education, Culture, Sports, Science and Technology, Government of Japan, 2011-2014.
Achieved 3rd Place of Int'l. RoboCup 2005 Competitions, Soccer Coach Simulation League, Member of Kasra Team, Osaka, Japan, 2005.
Recognized as an Exceptional Talent during Master's Program by Amirkabir University of Technology, 2010 (Ranked 1st in Fall 2008 & 2nd in Spring 2008 semesters in Artificial Intelligence Dept.)

Affiliations

IEEE Member (2009~), IEEE Robotics and Automation Society (2010~), Signal Processing Society (2015~), IEEE Systems, Man, and Cybernetics Society (2017~)
CALICO (Computer-Assisted Language Instruction Consortium), 2015~
Robocup Soccer Simulation Team Nemesis (2D Sim, 2010), Aria 3D (3D Sim, 2006~2008), and Kasra (2D Coach Sim, 2005)
CIPPRS (Canadian Image Processing and Pattern Recognition Society) Member, 2017~
IEICE (Institute of Electronics, Information and Communication Engineers) Member, 2017~

Services

Served as a reviewer for IEEE Trans. Of Multimedia, ACM Computing Surveys (CSUR), Journal of Neural Computing and Applications (NCAA), MDPI Journals (Sensors, Future Internet, Electronics, Information) Journal, IEICE Tran. on Information and Systems, Journal of Computer and Communications (JCC), Journal of ICT Research and Applications, and ACL’23, AAAI’23, AACL-IJNLP’22, ACCV’22, COLING’22, CVPR’20, ICML’20, ECCV’20, WiNLP’20, ACL’20, IJCNLP’20, WiNLP’19, ICAIR’16, ICCTS’17 ISCAI’18, FSDM’18, SNSP’18 conferences (Publons page: Top Reviewer of 2019)
Organizing Committee of Widening NLP Workshop, WiNLP’2020, Seattle, USA, 2020;
Organizing Committee of Widening NLP Workshop, WiNLP’2020, Seattle, USA, 2020;
Editorial Member of Journal of Robotics Engineering and Automation Technology, Gavin Publishers, USA, 2017-present.
Editorial Member of Progress in Human Computer Interaction Journal, Whioce Publishing Pte. Ltd., 2018-present.
Organizing Committee of Int’l Conference on Systems and Informatics, ICSAI’17, Hangzhou, China, 2017.
Organizing Committee of Int’l Conference on Computing Intelligence and Information System, CIIS’17, Nanjing, China, 2017.
Organizing Committee of Int’l Conference on Computer Science and Technology, ICCST’17, Guilin, China, 2017.
Scientific Committee of Intl. Computer Society of Iran Computer Conference, IKT'20, Tehran, Iran, 2020.
Organizing Committee of Intl. Computer Society of Iran Computer Conference, CSICC'09, Tehran, Iran, 2009.
Organizing Committee of 1st and 2nd Conf. of Information & Knowledge Technology, IKT 2003 & IKT 2005, Tehran, Iran, 2003-2005.

Projects

Efficient Diverse Ensemble for Discriminative Co-Tracking (DEDT)

An active co-tracker with self-organizing committee of classifiers, considering diversity for model update

Adversarial Ensemble Co-Tracker (ACET)

An active self-correcting committee of classifiers to perform collaborative tracking

Deep Q-Learning for Correlation Filter Tracking (DQCF)

A correlation filter tracker with reinforcement-learning-tuned learning rate

Query-by-Boosting Tracker (QBST)

a committee of weak classifiers in a Boosting framework

Collective Memory Tracker (CMT)

a committee of classifiers with the same data but different memory spans

Query-by-Bagging Tracker (QBT)

a committee of classifiers with partial knowledge

Mimic Tracker (MIMIC)

Aenean ornare velit lacus, ac varius enim lorem ullamcorper dolore. Proin aliquam facilisis ante interdum. Sed nulla amet lorem feugiat tempus aliquam.

Information-Maximizing Sampling Tracker (IMST)

A novel sampling technique to provide most informative samples for the classifiers to learn the target/non-target ever-changing boundary

Data-Driven Probabilistic Occlusion Mask (OccMask)

Aenean ornare velit lacus, ac varius enim lorem ullamcorper dolore. Proin aliquam facilisis ante interdum. Sed nulla amet lorem feugiat tempus aliquam.

Expanding Histogram of Colors with Gridding (HOCx)

Aenean ornare velit lacus, ac varius enim lorem ullamcorper dolore. Proin aliquam facilisis ante interdum. Sed nulla amet lorem feugiat tempus aliquam.

Occlusion Aware Particle Filter Tracker (OAPFT)

Aenean ornare velit lacus, ac varius enim lorem ullamcorper dolore. Proin aliquam facilisis ante interdum. Sed nulla amet lorem feugiat tempus aliquam.

Adversarial Sampling Co-Tracker (ASCT)

An active co-tracker in which the main classifier is robustified against its adversarial examples by the assistance of the auxiliary detector

Partial and Synchronized Caption (PSC)

A novel method of captioning as a listening tool for the second language learners in which automatically selected set of words in a caption where words are synced to their corresponding speech signals. Click here (the official project page) to see demonstration videos on how PSC works.

Long and Short Memory Balancing in Visual Co-Tracking using Q-Learning (QACT)

An adaptive active-co tracker, that uses Q-learning to balance the short vs. long memory usage and speed vs. accuracy trade-off

Get in touch

kourosh.meshgi [at] riken.jp
RIKEN AIP, Mitsui Building 15F, 1-4-1 Nihonbashi,Chuo-ku, Tokyo