An AI/ML Paper Recommendation Newsletter, Robotic Assembly Using Deep Reinforcement Learning & Language Modeling tool using Shorter Inputs in today’s Data Science Daily 📰
Here’s a weekly newsletter with AI/ML paper recommendations, personalised to your interests.
- 🦄 Publications from top institutes and journals.
- 🤖 Personalised to your interests.
- 📨 Delivered straight to your inbox.
Subscribe here 👉 www.serendipity.so
The goal of this tutorial is to show how you can apply DRL to solve your own robotic challenge. One of the most exciting advancements, that has pushed the frontier of the Artificial Intelligence (AI) in recent years, is Deep Reinforcement Learning (DRL).
DRL belongs to the family of machine learning algorithms. It assumes that intelligent machines can learn from their actions similar to the way humans learn from experience. Over recent years we could witness some impressive real-world applications of DRL. The algorithms allowed for major progress especially in the field of robotics. If you are interested in learning more about DRL, we encourage you to get familiar with the exceptional Introduction to RL by OpenAI.
GitHub: https://github.com/arrival-ltd/catalyst-rl-tutorial
Watch the tutorial here ▶ https://youtu.be/1BG5pC8WbTE
Blog: https://medium.com/pytorch/robotic-assembly-using-deep-reinforcement-learning-dfd9916c5ad7
Shortformer trains 65% faster, is 9x faster at token-by-token generation (as is done when sampling from GPT-3) and achieves better perplexity than the baseline model. This is achieved by combining two methods:
- In the first method (Staged Training), LM on short sequence is first trained before moving on to longer ones. This not only saves a lot of time (during training) but it also surprisingly improves perplexity.
- In the second method, Position-Infused Attention, absolute positions to the keys/queries are added instead of to word vectors. This enables in training a recurrent (TransformerXL-like) LM very efficiently. ere it again shows that shorter inputs here perform better than longer ones.
Here’s the code 👉 https://github.com/ofirpress/shortformer
Reach out to us on community@jovian.ai to get featured here. Learn data science and machine learning with free hands-on data science courses on Jovian.
Follow us on Twitter, LinkedIn, and YouTube to stay updated.