• Variational Autoencoder Learning Notes

• Reference Article:

  • Auto-Encoding Variational Bayes
  • Daniel Daza, The Variational Autoencoder
  • Sūshén's VAE series

• On VAE, the original paper and the two blogs above have already explained it very clearly. I am just repeating and paraphrasing, just to go through it myself. If anyone reads this blog, I recommend reading these three reference sources first

• Record the mathematical derivation of GloVe word vectors, as the original paper does not derive the model graphically but rather calculates the objective function through pure mathematical operations. This design approach is very interesting, and it also writes out and compares the mathematical essence of word2vec.
• GloVe: Global Vectors for Word Representation

• Convolutional Sequence to Sequence

• Robust Unsupervised Cross-Lingual Word Embedding Mapping

Course Notes on Computational Linguistics, Reference Textbook: Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition.

Note for John Mount's "The Equivalence of Logistic Regression and Maximum Entropy Models" and explains that this proof is a special case of the general derivation proof of the maximum entropy model introduced in statistical learning methods

Conclusion

• Maximum entropy model is softmax classification
• Under the balanced conditions of the general linear model, the model mapping function that satisfies the maximum entropy condition is the softmax function
• In the book on Statistical Machine Learning methods, a maximum entropy model defined under the feature function is presented, which, along with softmax regression, belongs to the class of log-linear models
• When the feature function extends from a binary function to the feature value itself, the maximum entropy model becomes a softmax regression model
• The maximum entropy maximizes conditional entropy, not the entropy of conditional probabilities, nor the entropy of joint probabilities.

Attended a light salon at Tsinghua University's FIT, which introduced some advancements in machine reading comprehension. Interestingly, the PhD who spoke at 9 am also mentioned an unpublished work: BERT, which is very impressive and well-funded; it took eight p100 GPUs to train for a year. By 10:30, Machine Intelligence had already published a report, and by the afternoon, Zhihu was buzzing with discussions, saying that a new era for NLP had arrived... This salon is part of a series, and there may be future sessions on machine translation, deep Bayesian, transfer learning, and knowledge graphs, so if you have the time, you might as well listen and take notes.

Deep-Bayes 2018 Summer Camp的习题 填不动了，就到这吧

Record the principles and derivations of algorithms used for inferring unknown variables in probabilistic machine learning, such as Variational Inference, Expectation Maximization, and Markov Chain Monte Carlo. Many contents and derivations, as well as images, come from the online course and lecture notes of Professor Xu Yida at the University of Technology Sydney. Professor Xu's series of videos on non-parametric Bayesian methods are very good, and you can find the videos by searching his name on Bilibili or Youku. The address of Professor Xu's course notes is roboticcam/machine-learning-notes. Unless otherwise specified, some screenshots and code are from Professor Xu's lecture notes. Other contents come from various books or tutorials, and the references will be indicated in the text.

The ten major algorithms of statistical learning methods have been simplified and handwritten out (although I think the book itself is already quite concise). Now there is only the process of the algorithms themselves; in the future, if I have any new understandings, I will supplement them. The writing is ugly, even I can't bear to look at it, so I post it purely as a backup

Latent Dirichlet Allocation Document Topic Generation Model Study Notes This article mainly summarizes from "Mathematical Curiosities of LDA(LDA数学八卦)," which is written very beautifully (recommend reading the original first). There are many places that spark further thought, and this article sorts out the steps to derive LDA, removes some irrelevant extensions, and summarizes LDA in plain language.

A bachelor's graduation project involves developing a short sentence summarization model based on seq2seq and designing an emotional fusion mechanism. Now, let's provide a brief summary of the entire model

• Distractor Mechanism
• External Information Attention
• Pointer Copy Network PGNet
• Extractive Summary Based on RNN
• Transformer
• Selection gate mechanism

• Opening Work on Attention (Machine Translation)

• Luong attention, global and local attention,

• Opening Work on Attention (Automatic Text Summarization)

• Generative Summary Techniques Collection: LVT, Switching Networks, Hierarchical Attention

• Dialogue System, End-to-End Hierarchical RNN

• Weibo summary, supplement micropoints

• disan, directed transformer, attention mask

• Attention Extractor

• Generative Summary Based on Reinforcement Learning

• w2v, negative sampling

Recorded some basic knowledge of deep learning learned when recording the seq2seq model in the entry-level NLP.

算法刷题目录，方便自己查找回忆复习 之后(2018.9.27)只更新leetcode上的题了，也懒得整理源码了，leetcode上都存了，只记录思路吧

Introduction of the Lagrange multiplier method and its extension KKT conditions, as well as their applications in PCA and SVM

The school's innovation project has a simple app that implements the following functions: recording sound and saving it as a wav file, using JSON to communicate with the server, uploading the wav file to the server, converting it to a midi file on the server, downloading the midi file and sheet music from the server for playback. At the same time, the modified electronic piano can also communicate with the server, with the phone providing auxiliary parameters to the electronic piano, which reads the intermediate key value file of the music from the server via Arduino to play.

Notes on some concepts and algorithms in machine learning, sourced from:

• Elective Course on Pattern Recognition (An elective course for third-year students at Beijing University of Posts and Telecommunications, Pattern Recognition, textbook is "Pattern Recognition" compiled by Zhang Xuegong, published by Tsinghua University Press)
• Watermelon Book
• Statistical Learning Methods
• Deep Learning (Translated in Chinese: exacity/deeplearningbook-chinese)

Update:

• 2017-02-12 Overview Update

• 2017-03-01 Update k-Nearest Neighbors

• 2017-03-08 Update SVM

• 2018-01-04 Update of fundamental knowledge of machine learning and mathematical knowledge in the book "Deep Learning"

• 2018-08-09 The content of Statistical Learning Methods has been posted in another article titled "Handwritten Notes on Statistical Learning Methods," and it is estimated that it will not be updated anymore. Later, some remaining contents in "Deep Learning" may be updated

Lecture 17: Determinants and Their Properties

Determinant

• The determinant of matrix A is a number associated with the matrix, denoted as \(detA或者|A|\)

• Properties of determinants

  • \(detI=1\)

  • The sign of the determinant value will be reversed when rows are exchanged

  • The determinant of a permutation matrix is 1 or -1, depending on the parity of the number of rows exchanged

  • Two rows being equal makes the determinant equal to 0 (which can be directly deduced from property two)

  • Matrix elimination does not change its determinant (proof is below)

  • A certain row is 0, the determinant is 0 (multiplying by 0 is equivalent to a certain row being 0, resulting in 0)

  • When and only when A is a singular matrix

  • \(det(A+B) \neq detA+detB \\ detAB=(detA)(detB)\)

  • \(detA^{-1}detA=1\)

  • \(detA^2=(detA)^2\)

  • \(det2A=2^n detA\)

  • \(detA^T=detA\) (Proof see below)

Lecture 9: Linear Correlation, Basis, Dimension

Linear Correlation

• Background knowledge: Assume a matrix A, where m < n, i.e., the number of unknowns is greater than the number of equations. Therefore, in the null space, there are vectors other than the zero vector, up to m leading principal elements, and there exist n-m free vectors, and the entire equation system has non-zero solutions.
• Under what conditions is the vector \(x_1,x_2,x_3...x_n\) linearly independent? If there exists a combination of coefficients not all equal to zero such that the linear sum results in 0, then it is linearly dependent; otherwise, it is linearly independent.
• If there exists a zero vector in the set of vectors, then the set of vectors cannot be linearly independent.
• If three vectors are randomly drawn in two-dimensional space, they must be linearly dependent. Why? This can be deduced from background knowledge.
• For a matrix A, we are concerned with whether the columns are linearly dependent; if there exists a non-zero vector in the null space, then the columns are dependent.
• When \(v_1,v_2...v_n\) is the columns of A, if they are unrelated, then what is the null space of A? Only the zero vector. If they are related, then in addition to the zero vector, there exists a non-zero vector in the null space.
• When the column vectors are linearly independent, all column vectors are leading vectors, and the rank is n. When the column vectors are linearly dependent, the rank is less than n.

First Lecture: Geometric Interpretation of Systems of Equations

• From three perspectives to view the system of equations: row graph, column graph, matrix

Always wanted to build my own blog, previously thought of using WordPress, but suffered from laziness, didn't want to mess with the server. Later, I stumbled upon GitHub Pages, which automatically generates a website by uploading a js project, and everything is hosted on GitHub. The official instructions also recommend using this for writing blogs, so I started to try it. The general framework should be that GitHub Pages generates the website from your github.io project on GitHub, Hexo generates the static web page project from your blog content and custom settings, and then uploads it to your repository. To back up, we will establish two branches in the repository: one master for Hexo to upload static web page files, and one hexo for saving the local Hexo project. Below, I share some experiences and pitfalls encountered.

•   2017.2.8 update md writing software

•   2017.2.10 update mathjax cdn, add long gallery, update domain name, access分流(blog2.0)

•   2017.2.13 update optimization plugin, update top description, optimize long gallery, widen article width(blog3.0)

•   2017.3.30 update top description original address

•   2017.12.27 update异地恢复

•   2018.7.6 update a more comprehensive reference website

Introduction

Write an app to query the school's empty classrooms Pull information from the school's registration website, classify and display it, and add some miscellaneous things After all, it's my first time writing Android, so I want to try everything Download here: BuptRoom repository address: A simple Beiyou self-study room query system It took about 3 weekends to complete the first version, and then I spent about 1 month updating miscellaneous things After that, I spent about 1 month updating miscellaneous things Many things written in an unstandardized manner, and I just looked up and used them temporarily Summarize the experience of writing the App: