Analyzing (un)structured data to convert it into a structured, normalized format.

learn more… | top users | synonyms (1)

2
votes
2answers
5k views

Fastest C++ XML parsing library

I have thousands of .xml files from size 1MB-45MB (no DTDs). I need to parse and further manipulate these XML files before generating separate .xml files with the results of my regex. What the ...
34
votes
2answers
4k views

Do modern languages still use parser generators?

I was researching about the gcc compiler suite on wikipedia here, when this came up: GCC started out using LALR parsers generated with Bison, but gradually switched to hand-written ...
10
votes
5answers
30k views

Getting data from a webpage in a stable and efficient way

Recently I've learned that using a regex to parse the HTML of a website to get the data you need isn't the best course of action. So my question is simple: What then, is the best / most efficient and ...
10
votes
4answers
12k views

Generic file parser design in Java using the Strategy pattern

I am working on a product in which the responsibility of one of the modules is to parse XML files and dump the required content in a database. Even though the present requirement is only to parse XML ...
3
votes
2answers
167 views

Need overview of concepts and tools to translate a DSL to regular expressions

I'm looking for a little guidance. Until this morning, this was all over my head. After spending today researching Wikipedia, StackOverflow, etc., I'd say I've got my nose above the water. I'm tasked ...
7
votes
2answers
1k views

Language parsing to find important words

I'm looking for some input and theory on how to approach a lexical topic. Let's say I have a collection of strings, which may just be one sentence or potentially multiple sentences. I'd like to ...
24
votes
6answers
2k views

What are the arguments against parsing the Cthulhu way?

I have been assigned the task of implementing a Domain Specific Language for a tool that may become quite important for the company. The language is simple but not trivial, it already allows nested ...
26
votes
8answers
4k views

Is it possible to statically predict when to deallocate memory---from source code only?

Memory (and resource locks) are returned to the OS at deterministic points during a program's execution. The control flow of a program by itself is enough to know where, for sure, a given resource can ...
10
votes
3answers
1k views

Writing a Compiler Compiler - Insight on Use and Features

This is part of a series of questions which focuses on the sister project to the Abstraction Project, which aims to abstract the concepts used in language design in the form of a framework. The ...
13
votes
2answers
3k views

What's the simplest example out there to explain the difference between Parse Trees and Abstract Syntax Trees?

To my understanding, a parser creates a parse tree, and then discards it thereafter. However, it can also pop out an abstract syntax tree, which the compiler supposedly makes use of. I'm under the ...
13
votes
5answers
4k views

Are separate parsing and lexing passes good practice with parser combinators?

When I began to use parser combinators my first reaction was a sense of liberation from what felt like an artificial distinction between parsing and lexing. All of a sudden everything was just ...
11
votes
5answers
11k views

Can the csv format be defined by a regex?

A colleague and I have recently argued over whether a pure regex is capable of fully encapsulating the csv format, such that it is capable of parsing all files with any given escape char, quote char, ...
1
vote
4answers
617 views

Is it possible to create a single tokenizer to parse this?

This extends off this other Q&A thread, but is going into details that are out of scope from the original question. I am generating a parser that is to parse a context-sensitive grammar which can ...
4
votes
4answers
1k views

What is the responsibility or benefit of a Tokenizer?

Suppose I had a grammar like: object { members } members pair pair string : value value number string string " chars " chars char char chars number digit ...
3
votes
2answers
652 views

Process to generate hierarchical structure based on relational data?

I have a csv of employee ids, names, and a reference column with the id of their direct manager, say something like this emp_id, emp_name, mgr_id 1,The Boss,, 2,Manager Joe,1 3,Manager Sally,1 ...
3
votes
2answers
4k views

How to get lookahead symbol when constructing LR(1) NFA for parser?

I am reading an explanation (awesome "Parsing Techniques" by D.Grune and C.J.H.Jacobs; p.292 in the 2nd edition) about how to construct an LR(1) parser, and I am at the stage of building the initial ...
2
votes
2answers
144 views

Algorithm to go from infix notation to a tree

I've been trying to figure out an algorithm to go from an infix equation to a syntax tree, like so: (1+3)*4+5 + * 5 + 4 1 3 However, I don't just want it to handle operators, I ...
0
votes
1answer
68 views

What is latent parsing in NLP?

I'm reading a paper describing a NLP work, and I hardly catch the concept of a term, "latent parsing". Original paper: http://web.stanford.edu/~angeli/papers/2013-acl-temporal.pdf The figure in the ...
-3
votes
2answers
299 views

What is the name of a grammar which can change its tokenizer in mid parse?

I was creating a language and discovered that my language tokenizer would have to change depending where in the parse it is. I.e. abc[1] would be parsed as 4 tokens (abc, [, 1, ]), where as { abc[1] ...