Natural language processing draws knowledge from a diverse collection of fields including computer science, linguistics, and statistics in order to extract pertinent information from the spoken or written word.

learn more… | top users | synonyms

136
votes
14answers
8k views

Simple method for reliably detecting code in text?

GMail has this feature where it will warn you if you try to send an email that it thinks might have an attachment. Because GMail detected the string see the attached in the email, but no actual ...
16
votes
2answers
531 views

How to find hard to misspell given names?

Here is a question that I believe could be solved with some data mining and a sophisticated algorithm, but I don't quite know how. Any pointers as to what data sources to use and what algorithm to ...
11
votes
6answers
2k views

How to teach a script to detect sarcasm? [closed]

I'm currently building a fun script, that basically matches given phrases and gives a predefined response based on the match-points. You can ask it to retrieve some information based on live feeds, ...
10
votes
2answers
1k views

Persisting natural language processing parsed data

I've recently started experimenting with natural language processing (NLP) using Stanford's CoreNLP, and I'm wondering what are some of the standard ways to store NLP parsed data for something like a ...
8
votes
2answers
693 views

How do personal assistants typically generate sentences?

This is sort of a follow up to this question about NLG research directions in the linguistics field. How do personal assistant tools such as Siri, Google Now, or Cortana perform Natural Language ...
7
votes
3answers
4k views

What algorithm(s) can be used to achieve reasonably good next word prediction?

What is a good way of implementing "next-word prediction"? For example, the user types "I am" and the system suggests "a" and "not" (or possibly others) as the next word. I am aware of a method that ...
7
votes
2answers
167 views

Guess if a time is AM or PM

I'm currently in the process of writing a human date parser. By human date, I mean it should be able to interpret strings as "tomorrow at 2" and return a valid date depending on the current time. The ...
6
votes
1answer
174 views

Identifying plagiarized jokes? [closed]

I'd like to be able to identify duplicate jokes posted on a website. I can build up a reasonably large database of previously-posted jokes, and then I'd like to look at each new joke as it comes in ...
4
votes
6answers
2k views

Are there any algorithms for splitting or combining words into their more common form?

Are there any existing algorithms which can look through a list of words and split or combine words into their more common form? For example, I have a list of many business names in the health care ...
4
votes
1answer
5k views

Starting point for learning natural language processing in Java [closed]

Currently I am a high school student and new to the natural language processing field. I am trying to write a Java program that can provide answers that can be understood by humans to the questions ...
4
votes
1answer
138 views

How can I test a search engine for an uncommon human language?

We are writing a search engine from scratch in a quite uncommon language, Aramaic, mostly for learning purposes but also because few resources are available in given language. The engine is/will be ...
3
votes
4answers
3k views

Programming a chatterbot - understanding language [closed]

I hope this is an answerable question. Let me give it some context: I am a psychology student and a programmer. I'm going to look into creating a artificial intelligence in the form of a chatterbot. ...
3
votes
1answer
282 views

Sentence Tree vs. Words List

I was recently tasked with building a Name Entity Recognizer as part of a project. The objective was to parse a given sentence and come up with all the possible combinations of the entities. One ...
3
votes
2answers
4k views

Separating words in a string

How do I separate words in a string? In the following I have a random sample of words in a string extracted from text file with over a million words. Here's the string: "intervene Pockets ...
3
votes
3answers
711 views

NLP - Queries using semantic wildcards in full text searching, maybe with Lucene?

Let's say I have a big corpus (for example in english or an arbitrary language), and I want to perform some semantic search on it. For example I have the query: "Be careful: [art] armada of [sg] is ...
3
votes
1answer
216 views

Tasks incorrectly categorized with online text classifiers

Context: Finding company to do the job The use case of an application I want to write is that a user will describe a task to be performed by some company, e.g., I want to move 10 boxes (...
3
votes
2answers
406 views

Available options for classifying words in text?

I am researching ways to classify words in text and I'm wondering what options there are and which are best suited to this job. I'm mostly interested in keywords which are most often nouns. So far I ...
2
votes
2answers
76 views

How to create an efficient implementation of a dictionary of synonyms?

How do I create a dictionary of synonyms that is efficient in terms of retrieving synonyms of a word? Basically, I want to use some sorts of data storage to store groups of words, and enable an end-...
2
votes
1answer
274 views

How can I distinguish between one-worded expressions and two-worded expressions?

I was told to create two lists concerning most frequently used words from a plain text (10 Mb arbitrary texts) as monograms (for single worded expressions such as human, water, is) and bigrams (for ...
1
vote
4answers
1k views

How to automatically connect letters into words? [closed]

I would like to connect letters into meaningful words? E.g. input: T E ST IN G WE B A P P LIC AT IO N S output: TESTING WEB APPLICATIONS what kind of task is this? A task in NLP, or just some ...
1
vote
1answer
904 views

Is Agile Development used in Machine Learning and Natural Language Processing?

I've been developing web apps for a while now and it is standard practice in our team to use agile development techniques and principles to implement the software. Recently, I've also become involved ...
1
vote
2answers
1k views

How to create a Semantic Network like wordnet based on Wikipedia?

I am an undergraduate student and I have to create a Semantic Network based on Wikipedia. This Semantic Network would be similar to Wordnet(except for it is based on Wikipedia and is concerned with "...
1
vote
1answer
157 views

How to do simple natural language categorization [closed]

How would you go about parsing a sentence like "Bought two kilos of steak from Acme supermarket" into a data structure like the following JSON representation: { item: {name: "steak", tags: "meat,...
1
vote
1answer
98 views

What type of “spam filter” algorithm will tokenize characters for non-exact matches?

I was having a look at a basic example of spam filtering using a logistic regression algorithm and an answer posted on this Stack Overflow question: http://stackoverflow.com/questions/29291263/mllib-...
0
votes
2answers
190 views

How could an expert system get some meaning from a sentence? [closed]

How could an expert system tell following scenarios apart? How could we include some meaning of a sentence, if at all? A machine needs 1 day to produce x units, and, we know that two machines like ...
0
votes
1answer
263 views

Natural Language to Search Criteria - Date Ranges

Consider an application that stores a set of records that contain: Description Cost Purchase Date I'd like to be able to allow users to utilize natural language to search the dataset. For example,...
0
votes
1answer
27 views

Extract the main entity from a body of text for text categorisation?

I'm trying to categorise products based on various fields of data. I've had some success just matching search terms in the product names, but this naive approach doesn't work when it comes to larger ...
0
votes
1answer
349 views

What is the purpose of chunking strings in natural language processing?

I see many examples of libraries that use a 'chunker' and also people asking how to write chunkers, but what is this for and why do we need it? Isn't it enough to split a text by whitespace or other ...
0
votes
1answer
45 views

Which strategy to use in Client-Server messaging to trigger specific behavior in them?

I'm developing a virtual assistant to manage transactions in a business. I'm using an external service to handle Natural Language and get intents, actions and parameters from my user requests. The ...
0
votes
0answers
41 views

What algorithm to use for a specific 'Named Entity Recognition'/'Information extraction' problem

I am trying to write a model that will extract certain details from financial documents. It must be able to extract the; contract start date, contract duration, contract value and all named entities. ...
0
votes
0answers
22 views

What are some ways to go about automatically categorizing calendar data?

I'm working on an application that takes as input your calendar data (i.e. the event titles like "Lunch with Barney" and their respective times), and automatically assigns each of them to a category ...
0
votes
1answer
76 views

What is latent parsing in NLP?

I'm reading a paper describing a NLP work, and I hardly catch the concept of a term, "latent parsing". Original paper: http://web.stanford.edu/~angeli/papers/2013-acl-temporal.pdf The figure in the ...
-3
votes
1answer
86 views

Using NLP can we create a summary of functional specification [closed]

Just thinking that is it somehow possible by using Language processing APIs to extract a must have points or an intelligent summary from documents like functional specifications ..