Block Sparse Matrices for Smaller and Faster Language Models
【RSS 原文 · 未经 AI 筛选或翻译】此订阅未提供摘要，请通过原文链接阅读。
#RSS原文
Hugging Face Blog
OpenAI News Generative language modeling for automated theorem proving
【RSS 原文 · 未经 AI 筛选或翻译】此订阅未提供摘要，请通过原文链接阅读。
OpenAI News Learning to summarize with human feedback
【RSS 原文 · 未经 AI 筛选或翻译】We’ve applied reinforcement learning from human feedback to train language models that are better at summarization.
OpenAI News OpenAI Scholars 2020: Final projects
【RSS 原文 · 未经 AI 筛选或翻译】Our third class of OpenAI Scholars presented their final projects at virtual Demo Day, showcasing their research results from over the past five months.
Hugging Face Blog The Reformer - Pushing the limits of language modeling
【RSS 原文 · 未经 AI 筛选或翻译】此订阅未提供摘要，请通过原文链接阅读。
OpenAI News Procgen and MineRL Competitions
【RSS 原文 · 未经 AI 筛选或翻译】We’re excited to announce that OpenAI is co-organizing two NeurIPS 2020 competitions with AIcrowd, Carnegie Mellon University, and DeepMind, using Procgen Benchmark and MineRL.
OpenAI News Image GPT
【RSS 原文 · 未经 AI 筛选或翻译】We find that, just as a large transformer model trained on language can generate coherent text, the same exact model trained on pixel sequences can generate coherent image completions and samples. By establishing a correlation between sample quality and image classification accuracy, we show that our best generative model also contains features competitive with top convolutional nets in the unsupervised setting.
OpenAI News OpenAI API
【RSS 原文 · 未经 AI 筛选或翻译】We’re releasing an API for accessing new AI models developed by OpenAI.
OpenAI News Language models are few-shot learners
【RSS 原文 · 未经 AI 筛选或翻译】此订阅未提供摘要，请通过原文链接阅读。
OpenAI News AI and efficiency
【RSS 原文 · 未经 AI 筛选或翻译】We’re releasing an analysis showing that since 2012 the amount of compute needed to train a neural net to the same performance on ImageNet classification has been decreasing by a factor of 2 every 16 months. Compared to 2012, it now takes 44 times less compute to train a neural network to the level of AlexNet (by contrast, Moore’s Law would yield an 11x cost improvement over this period). Our results suggest that for AI tasks with high levels of recent investment, algorithmic progress has yielded more gains than classical hardware efficiency.
OpenAI News Jukebox
【RSS 原文 · 未经 AI 筛选或翻译】We’re introducing Jukebox, a neural net that generates music, including rudimentary singing, as raw audio in a variety of genres and artist styles. We’re releasing the model weights and code, along with a tool to explore the generated samples.
OpenAI News Improving verifiability in AI development
【RSS 原文 · 未经 AI 筛选或翻译】We’ve contributed to a multi-stakeholder report by 58 co-authors at 30 organizations, including the Centre for the Future of Intelligence, Mila, Schwartz Reisman Institute for Technology and Society, Center for Advanced Study in the Behavioral Sciences, and Center for Security and Emerging Technologies. This report describes 10 mechanisms to improve the verifiability of claims made about AI systems. Developers can use these tools to provide evidence that AI systems are safe, secure, fair, or privacy-preserving. Users, policymakers, and civil society can use these tools to evaluate AI development processes.
OpenAI News OpenAI Microscope
【RSS 原文 · 未经 AI 筛选或翻译】We’re introducing OpenAI Microscope, a collection of visualizations of every significant layer and neuron of eight vision “model organisms” which are often studied in interpretability. Microscope makes it easier to analyze the features that form inside these neural networks, and we hope it will help the research community as we move towards understanding these complicated systems.
Hugging Face Blog How to generate text: using different decoding methods for language generation with Transformers
【RSS 原文 · 未经 AI 筛选或翻译】此订阅未提供摘要，请通过原文链接阅读。
Hugging Face Blog How to train a new language model from scratch using Transformers and Tokenizers
【RSS 原文 · 未经 AI 筛选或翻译】此订阅未提供摘要，请通过原文链接阅读。
OpenAI News OpenAI standardizes on PyTorch
【RSS 原文 · 未经 AI 筛选或翻译】We are standardizing OpenAI’s deep learning framework on PyTorch.
OpenAI News Scaling laws for neural language models
【RSS 原文 · 未经 AI 筛选或翻译】此订阅未提供摘要，请通过原文链接阅读。
OpenAI News Dota 2 with large scale deep reinforcement learning
【RSS 原文 · 未经 AI 筛选或翻译】此订阅未提供摘要，请通过原文链接阅读。
OpenAI News Deep double descent
【RSS 原文 · 未经 AI 筛选或翻译】We show that the double descent phenomenon occurs in CNNs, ResNets, and transformers: performance first improves, then gets worse, and then improves again with increasing model size, data size, or training time. This effect is often avoided through careful regularization. While this behavior appears to be fairly universal, we don’t yet fully understand why it happens, and view further study of this phenomenon as an important research direction.
OpenAI News Procgen Benchmark
【RSS 原文 · 未经 AI 筛选或翻译】We’re releasing Procgen Benchmark, 16 simple-to-use procedurally-generated environments which provide a direct measure of how quickly a reinforcement learning agent learns generalizable skills.
OpenAI News Safety Gym
【RSS 原文 · 未经 AI 筛选或翻译】We’re releasing Safety Gym, a suite of environments and tools for measuring progress towards reinforcement learning agents that respect safety constraints while training.
OpenAI News Benchmarking safe exploration in deep reinforcement learning
【RSS 原文 · 未经 AI 筛选或翻译】此订阅未提供摘要，请通过原文链接阅读。
OpenAI News GPT-2: 1.5B release
【RSS 原文 · 未经 AI 筛选或翻译】As the final model release of GPT-2’s staged release, we’re releasing the largest version (1.5B parameters) of GPT-2 along with code and model weights to facilitate detection of outputs of GPT-2 models. While there have been larger language models released since August, we’ve continued with our original staged release plan in order to provide the community with a test case of a full staged release process. We hope that this test case will be useful to developers of future powerful models, and we’re actively continuing the conversation with the AI community on responsible publication.
OpenAI News Solving Rubik’s Cube with a robot hand
【RSS 原文 · 未经 AI 筛选或翻译】We’ve trained a pair of neural networks to solve the Rubik’s Cube with a human-like robot hand. The neural networks are trained entirely in simulation, using the same reinforcement learning code as OpenAI Five paired with a new technique called Automatic Domain Randomization (ADR). The system can handle situations it never saw during training, such as being prodded by a stuffed giraffe. This shows that reinforcement learning isn’t just a tool for virtual tasks, but can solve physical-world problems requiring unprecedented dexterity.
OpenAI News OpenAI Scholars 2020: Applications open
【RSS 原文 · 未经 AI 筛选或翻译】We are now accepting applications for our third class of OpenAI Scholars.
OpenAI News Fine-tuning GPT-2 from human preferences
【RSS 原文 · 未经 AI 筛选或翻译】We’ve fine-tuned the 774M parameter GPT-2 language model using human feedback for various tasks, successfully matching the preferences of the external human labelers, though those preferences did not always match our own. Specifically, for summarization tasks the labelers preferred sentences copied wholesale from the input (we’d only asked them to ensure accuracy), so our models learned to copy. Summarization required 60k human labels; simpler tasks which continue text in various styles required only 5k. Our motivation is to move safety techniques closer to the general task of “machines talking to humans,” which we believe is key to extracting information about human values.
OpenAI News Emergent tool use from multi-agent interaction
【RSS 原文 · 未经 AI 筛选或翻译】We’ve observed agents discovering progressively more complex tool use while playing a simple game of hide-and-seek. Through training in our new simulated hide-and-seek environment, agents build a series of six distinct strategies and counterstrategies, some of which we did not know our environment supported. The self-supervised emergent complexity in this simple environment further suggests that multi-agent co-adaptation may one day produce extremely complex and intelligent behavior.
OpenAI News Testing robustness against unforeseen adversaries
【RSS 原文 · 未经 AI 筛选或翻译】We’ve developed a method to assess whether a neural network classifier can reliably defend against adversarial attacks not seen during training. Our method yields a new metric, UAR (Unforeseen Attack Robustness), which evaluates the robustness of a single model against an unanticipated attack, and highlights the need to measure performance across a more diverse range of unforeseen attacks.
OpenAI News GPT-2: 6-month follow-up
【RSS 原文 · 未经 AI 筛选或翻译】We’re releasing the 774 million parameter GPT-2 language model after the release of our small 124M model in February, staged release of our medium 355M model in May, and subsequent research with partners and the AI community into the model’s potential for misuse and societal benefit. We’re also releasing an open-source legal agreement to make it easier for organizations to initiate model-sharing partnerships with each other, and are publishing a technical report about our experience in coordinating with the wider AI research community on publication norms.
OpenAI News Learning Day
【RSS 原文 · 未经 AI 筛选或翻译】At OpenAI, each Thursday is Learning Day: a day where employees have the option to self-study technical skills that will make them better at their job but which aren’t being learned from daily work.
OpenAI News Microsoft invests in and partners with OpenAI to support us building beneficial AGI
【RSS 原文 · 未经 AI 筛选或翻译】Microsoft is investing $1 billion in OpenAI to support us building artificial general intelligence (AGI) with widely distributed economic benefits. We’re partnering to develop a hardware and software platform within Microsoft Azure which will scale to AGI. We’ll jointly develop new Azure AI supercomputing technologies, and Microsoft will become our exclusive cloud provider—so we’ll be working hard together to further extend Microsoft Azure’s capabilities in large-scale AI systems.
OpenAI News Why responsible AI development needs cooperation on safety
【RSS 原文 · 未经 AI 筛选或翻译】We’ve written a policy research paper identifying four strategies that can be used today to improve the likelihood of long-term industry cooperation on safety norms in AI: communicating risks and benefits, technical collaboration, increased transparency, and incentivizing standards. Our analysis shows that industry cooperation on safety will be instrumental in ensuring that AI systems are safe and beneficial, but competitive pressures could lead to a collective action problem, potentially causing AI companies to under-invest in safety. We hope these strategies will encourage greater cooperation on the safe development of AI and lead to better global outcomes of AI.
OpenAI News OpenAI Robotics Symposium 2019
【RSS 原文 · 未经 AI 筛选或翻译】We hosted the first OpenAI Robotics Symposium on April 27, 2019.
OpenAI News OpenAI Scholars 2019: Final projects
【RSS 原文 · 未经 AI 筛选或翻译】Our second class of OpenAI Scholars has concluded, with all eight scholars producing an exciting final project showcased at Scholars Demo Day at OpenAI.
OpenAI News OpenAI Fellows Fall 2018: Final projects
【RSS 原文 · 未经 AI 筛选或翻译】Our second class of OpenAI Fellows has wrapped up, with each Fellow going from a machine learning beginner to core OpenAI contributor in the course of a 6-month apprenticeship. We are currently reviewing applications on a rolling basis for our next round of OpenAI Fellows Summer 2019.
OpenAI News Transfer of adversarial robustness between perturbation types
【RSS 原文 · 未经 AI 筛选或翻译】此订阅未提供摘要，请通过原文链接阅读。
OpenAI News MuseNet
【RSS 原文 · 未经 AI 筛选或翻译】We’ve created MuseNet, a deep neural network that can generate 4-minute musical compositions with 10 different instruments, and can combine styles from country to Mozart to the Beatles. MuseNet was not explicitly programmed with our understanding of music, but instead discovered patterns of harmony, rhythm, and style by learning to predict the next token in hundreds of thousands of MIDI files. MuseNet uses the same general-purpose unsupervised technology as GPT-2, a large-scale transformer model trained to predict the next token in a sequence, whether audio or text.
OpenAI News Generative modeling with sparse transformers
【RSS 原文 · 未经 AI 筛选或翻译】We’ve developed the Sparse Transformer, a deep neural network which sets new records at predicting what comes next in a sequence—whether text, images, or sound. It uses an algorithmic improvement of the attention mechanism to extract patterns from sequences 30x longer than possible previously.
OpenAI News OpenAI Five defeats Dota 2 world champions
【RSS 原文 · 未经 AI 筛选或翻译】OpenAI Five is the first AI to beat the world champions in an esports game, having won two back-to-back games versus the world champion Dota 2 team, OG, at Finals this weekend. Both OpenAI Five and DeepMind’s AlphaStar had previously beaten good pros privately but lost their live pro matches, making this also the first time an AI has beaten esports pros on livestream.
OpenAI News OpenAI Five Finals
【RSS 原文 · 未经 AI 筛选或翻译】We’ll be holding our final live event for OpenAI Five at 11:30am PT on April 13.