Word embeddings: a workaround for training text focused artificial intelligence
In an awesome episode from Scorpion, Silvester goes to jail, Stallone's style. his goal: finding a scheme using the jail library, for criminal reasons, this is bad, pretty bad; not the crime, but using libraries, as sacred soil, for crimes. He was chosen for having a photograph memory. By the way, there is no evidence of photographic memory on reality! Let's stick to the facts! Silvester, I am a big fan, but here I must be realistic! 😁😂
The best we can do is creating artificial intelligence-based system to make this, or get at least close. The goal generally is not storing information, but mining them: our brain is not a hard disk, it is a data mining, a PCA machine. For instance, if you have a set of reviews, can you separate them into positive and negative? A bot can do it with TensorFlow.js !
As you may know by now, and if you do not, I am going tell you: artificial neural networks, even if they are awesome compared to classical methods, they still need you to give them things already chowed down things for them. In case of texts, you must find a best way to represent the text, after that, the neural networks will do its magic!
A way to represent text on neural networks (NNs) is called word embedding: see that if you are working with NNs, this can be hard if you are with a new problem, that no else solved. You will have to find a best representation as so the NNs will understand what you want to solve. Think like this: you need to explain you child that you need to go work, without telling them all the subtleties of adulthoods, all the complex math you do every day to make a work you hate!
In simple terms: you represent the words as grades of shades (i.e., colors, say, from white to black). After that, the NN will have no way to tell whether this is an image or a text. This is good, since we can use the same tool for several problems: you can use public libraries, such as TensorFlow.js. And this is the magic of NN compared to classical tools, which are generally tailored for the problem. NN are tailored for generical problems.
Tips for you: should you be doing say a PhD, make sure you either have a problem easy to represent on NNs, or make sure you know that it can be hard to find one. You may build an entire PhD pathway just for finding an optimal representation. No need to feel guilty if you have trouble finding one. In research, we just know the depth of the problem when we jump, and the free falling never ends!
Tips for you. The layer type in TensorFlow.js that allows you to perform word embedding is tf.layer.embedding().