SageMaker로 강화학습(RL) 마스터링 :: 남궁선 - AWS Community Day 2019

강화학습(RL) 마스터링 남궁선 - AWS Community Day 2019

SageMaker로 강화학습(RL) 마스터링 :: 남궁선 - AWS Community Day 2019 from AWS Korea UserGroup (AWS한국사용자모임)

SageMaker로 강화학습(RL) 마스터링 :: 남궁선 - AWS Community Day 2019

1. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. SageMaker (RL) SageMaker and DeepRacer Seon Namkung Student Hanyang University ERICA

2. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. (Reinforcement Learning) SageMaker RL Demo - SageMaker RL &Auto Scaling DeepRacer

3. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. (Reinforcement Learning)

4. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. …(1)

5. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. …(2)

6. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. (Reinforcement) . - Wikipedia -

7. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. “ ” - Richard S. Sutton and Andrew G. Barto, Reinforcement Learning: An Introduction -

8. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. inAI Artificial
 Intelligence Machine
 Learning Reinforcement
 Learning Supervised
 Learning Unsupervised
 Learning

9. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

10. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. (Environment) ?

11. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Rewards RL Algorithm

12. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

13. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. AUTONOMOUS CARS FINANCIAL TRADING DATACENTER COOLINGFLEET LOGISTICS

14. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. , RL … RL agent Environment

15. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon SageMaker RL

16. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon SageMaker Build - - - - , Train - - - - , Deploy - - API - -

17. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. RL RL toolkits that provide RL agent algorithm implementations RL-Coach RL-Ray RLLib Open AI Baselines … DQN PPO A3C Rainbow … DQN PPO A3CIMPALA … DQN PPO … Amazon SageMaker deep learning frameworks Tensorflow MxNet PyTorch Chainer SageMaker supported Customer BYO

18. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Common gym interfaces env.reset()
 env.render()
 env.step()
 env.action_space
 env.observation_space Simulation environments to model real-world problems AWS simulation 
 environments Open source 
 environments Custom developed environments Commercial simulators AWS RoboMaker EnergyPlus RoboSchool PyBullet … Bring Your Own SimuLink Amazon Sumerian

19. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Simulation environment RL 
 agent SageMaker Training R O L L O U T Initial model

20. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Actions and Observations Simulation environment Actions Observation RL 
 agent Initial model SageMaker Training R O L L O U T

21. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Training data Simulation environment Actions Observation RL 
 agent Training Data Observation, action, reward SageMaker Training

22. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Simulation environment Actions Observation RL 
 agent Model updates Training data Observation, action, reward SageMaker Training

23. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

24. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Simulation environment Actions Observation Trained Model RL 
 agent Model updates Training data Training visualization Observation, action, reward Deploy Model SageMaker Training

25. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. DEMO - SageMaker RL & Auto Scaling

26. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Example :Auto Scaling : • Environment : & • Action : / • State : ( , ) • Reward : R( , ) < >

27. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Configures

28. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. (Reward)

29. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Observation, Reward

30. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Training

31. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

32. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. DeepRacer

33. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS DeepRacer 1/18 ( )

34. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS DeepRacer

35. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. DeepRacer RL ACTION GOALAGENT ENVIRONMENT

36. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. AGENT STATE REWARD ENVIRONMENT ACTION VALUE FUNCTION POLICY FUNCTION

37. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon SageMaker RL +AWS RoboMaker accelerate learning forAWS DeepRacer

38. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Samples

39. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon SageMaker https://aws.amazon.com/ko/sagemaker/ Amazon SageMaker https://aws.amazon.com/ko/sagemaker/pricing/ Amazon SageMaker RL https://aws.amazon.com/ko/blogs/korea/amazon-sagemaker-rl- managed-reinforcement-learning-with-amazon-sagemaker/ Amazon SageMaker RL https://docs.aws.amazon.com/ko_kr/sagemaker/latest/dg/ sagemaker-rl-workflow.html

40. Thank you! © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Seon Namkung whitesoil.kr@gmail.com