Generating Word Embeddings from an Extreme Learning Machine for Sentiment Analysis and Sequence Labeling Tasks

Abstract

Word Embeddings Are Low-Dimensional Distributed Representations that Encompass a Set of Language Modeling and Feature Learning Techniques from Natural Language Processing (Nlp). Words or Phrases from the Vocabulary Are Mapped to Vectors of Real Numbers in a Low-Dimensional Space. in Previous Work, We Proposed using an Extreme Learning Machine (Elm) for Generating Word Embeddings. in This Research, We Apply the Elm-Based Word Embeddings to the Nlp Task of Text Categorization, Specifically Sentiment Analysis and Sequence Labeling. the Elm-Based Word Embeddings Utilizes a Count-Based Approach Similar to the Global Vectors (Glove) Model, Where the Word-Context Matrix is Computed Then Matrix Factorization is Applied. a Comparative Study is Done with Word2vec and Glove, Which Are the Two Popular State-Of-The-Art Models. the Results Show that Elm-Based Word Embeddings Slightly Outperforms the Aforementioned Two Methods in the Sentiment Analysis and Sequence Labeling Tasks.in Addition, Only One Hyperparameter is Needed using Elm Whereas Several Are Utilized for the Other Methods. Elm-Based Word Embeddings Are Comparable to the State-Of-The-Art Methods: Word2vec and Glove Models. in Addition, the Count-Based Elm Model Have Word Similarities to Both the Count-Based Glove and the Predict-Based Word2vec Models, with Subtle Differences.

Department(s)

Engineering Management and Systems Engineering

Comments

National Natural Science Foundation of China, Grant 61502338

Keywords and Phrases

Extreme learning machine (ELM); Global vectors (GloVe); Sentiment analysis; Sequence labeling; Text categorization; Word embeddings; Word2Vec

International Standard Serial Number (ISSN)

1866-9964; 1866-9956

Document Type

Article - Journal

Document Version

Citation

File Type

text

Language(s)

English

Rights

© 2024 Springer, All rights reserved.

Publication Date

01 Aug 2018

Link to Full Text

Share

 
COinS