Projects
Multi-Agent Reinforcement Learning
Multi-Agent Reinforcement Learning (MARL) is a method that uses reinforcement learning techniques to teach multiple agents to cooperate, compete, or perform other actions through trial and error within the same environment.
Optimization of Delivery Planning in Non-Grid Map by Multi-Agent Reinforcement Learning
support by joint research with Panasonic Corporation (Principal Investigator, October 2022~)
Abstract: Optimal routing for multi-drone delivery can be formalized as the Multi-agent Path Finding (MAPF) problem, where multiple agents are controlled to avoid collisions while minimizing the overall cost. However, many existing MAPF methods assume grid-based maps, making them inefficient when applied directly to non-grid maps based on actual urban areas, as addressed in this research. In this paper, we propose a method that combines multi-agent reinforcement learning with exploratory techniques. Simulation experiments show that, compared to conventional methods, our approach improves learning efficiency and the rate of successful goal arrivals. The results have been accepted and presented at prominent international conferences in the AI and agent domains: PAAMS-2022 (CORE B) and PRICAI-2023 (CORE B).
Publications:
Shiyao Ding, Hideki Aoyama, and Donghui Lin, “Combining Multiagent Reinforcement Learning and Search Method for Drone Delivery on a Non-Grid Graph,” 20th International Conference on Practical Applications of Agents and Multi-Agent Systems (PAAMS 2022), L’Aquila, Italy, July, 2022.
Shiyao Ding, Hideki Aoyama and Donghui Lin, MARL4DRP: Benchmarking Cooperative Multi-Agent Reinforcement Learning Algorithms for Drone Routing Problems, 20th Pacific International Conference on Artificial Intelligence (PRICAI 2023), November 17-19, 2023, Jakarta, Indonesia.
Cooperative Large-Scale Edge Cloud Computing by Multi-Agent Reinforcement Learning
supported by Japan Science and Technology Agency (JST) Next Generation Researcher Challenging Research Program ( Research Fellow, October 2021 ~ September 2022)
Abstract: With the recent surge in IoT devices, the number of servers in edge clouds has increased, drawing attention to methods that allow servers to cooperate to meet computational resource demands. Traditional studies have proposed competitive systems managed by self-interested servers, which have not been able to fully satisfy the demands for computational resources. Moreover, these servers are distributed across multiple regions and are unable to reference information from other servers, making the learning process unstable and cooperation less effective. In this research, we propose a cooperative control platform for large-scale edge clouds using multi-agent reinforcement learning. We introduce several new multi-agent reinforcement learning techniques, achieving low-latency and high-quality cooperation between servers. This paper has been accepted and highly regarded, presented at the top-tier international conference on service computing, IEEESCC-2020 (CORE A), and the 20th Information Science Forum (FIT2021) top conference session.
Publications:
Shiyao Ding and Donghui Lin, “Dynamic Task Allocation for Cost-Efficient Edge Cloud Computing,” The 17th IEEE International Conference on Services Computing (IEEE SCC 2020), pp.218-225, Beijing, China, October, 2020. [DOI]
Shiyao Ding. Multi-Agent Reinforcement Learning for Task Allocation in Cooperative Edge Cloud Computing. International Conference on Service-Oriented Computing (ICSOC 2021), PhD Symposium, November 2021.
Automated Negotiating Agents
Automated negotiating agents act on behalf of humans in the processes of collaboration and negotiation, taking over tasks that humans may find tedious or processing large amounts of information that may be beyond human capability.
Automated Negotiating Agent for Consensus Building with Humans by Multi-Agent Reinforcement Learning
supported by the Japan Society for the Promotion of Science (JSPS) Grant-in-Aid for Scientific Research (C) (Principal Investigator, April 2023~)
Abstract: This research aims to develop an automatic negotiation agent capable of forming agreements with humans and to validate its utility in real-world applications. In this study, we employ multi-agent reinforcement learning techniques, taking into account the characteristics related to human's bounded rationality, to develop an automatic negotiation agent that can form agreements within a few rounds of negotiation. Based on this, we will construct a negotiation service platform between agents and humans. Its effectiveness will then be verified through pilot experiments in real-world scenarios, such as online sales of electronic products.
Publications:
Shiyao Ding and Takayuki Ito, A Deep Reinforcement Learning Based Facilitation Agent for Consensus Building among Multi-Round Discussions, 20th Pacific International Conference on Artificial Intelligence (PRICAI 2023), November 17-19, 2023, Jakarta, Indonesia.
support by Japan Science and Technology Agency (JST) CREST（Participant，Oct. 2020~）
Abstract：This project realizes a SNS platform called “hyper democracy” where softwaer agents and human collaborately make democratic decisions. Here, agents who work on behalf of human are allocated into SNS distributedly, and support consensus making process (social multiagent system). These agens protect users from the problems like flaming, fake news, etc., while supporing smarter consensus and group decision making.
Publications:
Shiyao Ding and Takayuki Ito, Self-Agreement: A Framework for Fine-tuning Language Models to Find Agreement among Diverse Opinions, 20th Pacific International Conference on Artificial Intelligence (PRICAI 2023), November 17-19, 2023, Jakarta, Indonesia.
supported by Japan Science and Technology Agency (JST) AIP Challenging Program ( Principal Investigator, June 2022 ~ March 2023)
Abstract：Despite the recent success of applying graph neural networks (GNN) for argumentation mining tasks like argument classifications, generalization of cross-topic remains an open problem. This is because each topic-based argumentation data corresponds to a specific argumentation graphs and could make the trained GNN models completely fail in the new argumentation graphs. To solve this issue, a meta-learning based GNN method is proposed, where the meta-learner is trained with considering all specific topic-based GNN models and allows it to generalize to new argumentation graphs fastly.
Publications:
Shiyao Ding and Takayuki Ito. Graph Convolutional Networks for Link Prediction in Argument Structure Extraction. KICSS-2022.