Google’s AI language model Reformer can process the entirety of novels

Whether it’s language, music, speech, or video, sequential data isn’t easy for AI and machine learning models to comprehend – particularly when there’s dependence on extensive surrounding context.

That’s how all AI models extract features and learn to make predictions, but Transformer uniquely have attention such that every output element is connected to every input element.

As my colleague Khari Johnson notes, one of the biggest machine learning trends of 2019 was the continued growth and proliferation of natural language models based on this Transformer design.

Google open-sourced BERT, a Transformer-based model, in 2018.

The research team experimented with Reformer-based models on images and text, using them to generate missing details in images and process the entirety of Crime and Punishment.

“They leave to future work applying them to even longer sequences and improving their handling of positional encodings. We believe Reformer gives the basis for future use of Transformer models, both for long text and applications outside of natural language processing,” added Kaiser and Kitaev.

In an interview late last year, Google AI chief Jeff Dean told VentureBeat that larger context would be a principal focus of Google’s work going forward.

This article was summarized automatically with AI / Article-Σ ™/ BuildR BOT™.

Original link

How I used Deep Learning to Optimize an Ecommerce Business Process with Keras

Nowadays in the era of deep learning and computer vision, checking manually web content is considered as a flaw and very time consuming, furthermore it can lead to many mistakes such as this one below, where moderators had accepted a laptop ad in phone category which is wrong and affect search engine quality, while this work could be done in a second by a Deep Learning model.

In this blog post I will cover how I optimized this process by building a simple Convolutional Neural Network using Keras framework, that can classify if an uploaded image is for a phone or a laptop and tell us if the image is matching the ad category or not.

2.2 Image resizingThis step is absolutely depending on the adopted Deep Learning architecture, for example when using Alexnet model to classify images, the input shape should be 227 x 227, while for VGG-19 the input shape is 224 x 224.Since we are not going to adopt any pre-built architecture, we will build our own Convolutional Neural Network model, where the input size is 64 x 64, like shown in the code snapshot below.

For this model, we will discuss each component how it was implemented using Keras and its own parameters starting from convolutions to fully connected layer, but first of all, let’s discover the full architecture of the built-in model.

We have to compile the network that we have just built by calling compile function, it is a mandatory step for every model built using Keras.

Analyzing Model with TensorBoardIn this step, we will see how we can analyse our model behavior using TensorBoard.

ConclusionTo conclude with, this blog post shows a complete computer vision pipeline by building a Deep Learning model that can predict the class of an uploaded image applied on eCommerce context, starting from the Data Collecting to the Data Modeling and finishing by Model Deployment as a web app.

This article was summarized automatically with AI / Article-Σ ™/ BuildR BOT™.

Original link