Analyzing (un)structured data to convert it into a structured, normalized format.

learn more… | top users | synonyms (1)

0
votes
0answers
20 views

Design pattern for XML parsing on different files

I have two XML files with different structure. I have written two SAX Handlers for it. I am creating these handlers object from a utility class. public class Util { public static void parseFile1() ...
6
votes
2answers
341 views

In layman's terms, what is left recursion?

According to one page on code.google.com, "left recursion" is defined as follows: Left recursion just refers to any recursive nonterminal that, when it produces a sentential form containing ...
0
votes
1answer
44 views

What is latent parsing in NLP?

I'm reading a paper describing a NLP work, and I hardly catch the concept of a term, "latent parsing". Original paper: http://web.stanford.edu/~angeli/papers/2013-acl-temporal.pdf The figure in the ...
3
votes
1answer
215 views

Why did GCC switch from Bison to a recursive descent parser for C++ and C?

Was there a language change that required it or some practical reason why Bison was no longer appropriate or optimal? I saw on wikipedia that they switched, referring to the GCC 3.4 and GCC 4.1 ...
2
votes
3answers
109 views

How exactly is an Abstract Syntax Tree created?

I think I understand the goal of an AST, and I've built a couple of tree structures before, but never an AST. I'm mostly confused because the nodes are text and not number, so I can't think of a nice ...
3
votes
1answer
49 views

How to parse different number types with LALR(1)

Consider a LALR(1) parser for a file format that allows integer numbers and floating point numbers. As usual, something like 42 shall be a valid integer and a valid float (with some automagic ...
1
vote
1answer
21 views

Rule order for Parsing Lists with LALR(1)

When creating the grammar for parsing a list (something like “ITEM*”) with a LALR(1) parser, this basically can be done in two ways: list : list ITEM | ; or list : ITEM list | ...
22
votes
2answers
1k views

Do modern languages still use parser generators?

I was researching about the gcc compiler suite on wikipedia here, when this came up: GCC started out using LALR parsers generated with Bison, but gradually switched to hand-written ...
0
votes
2answers
68 views

Large csv's to html report

I am working on a web front end + front end services. I receive good sized csv files (10k lines). My service processes them and condenses them into one larger csv file (up to 300k lines). This ...
3
votes
2answers
98 views

Need overview of concepts and tools to translate a DSL to regular expressions

I'm looking for a little guidance. Until this morning, this was all over my head. After spending today researching Wikipedia, StackOverflow, etc., I'd say I've got my nose above the water. I'm tasked ...
-1
votes
1answer
46 views

How to keep AST for feature access?

Consider such code (let's say it is C++) Foo::Bar.get().X How one should keep the AST for this -- as "tree" with root at left Foo(Bar(get(X)), or with root at right (((Foo)Bar)get)X? Or maybe as a ...
1
vote
3answers
162 views

Parsing mathematical expressions with two values that have parentheses and minus signs

I'm trying to parse equations like the ones below, which only have two values or the square root of a certain value, from a text file: 100+100 -100-100 -(100)+(-100) sqrt(100) by the minus ...
0
votes
0answers
32 views

Parsing a website's source [duplicate]

I want to create an application and maybe upload it to the play store but I am not sure if that what my app does is legal or not. I am downloading a page's source from a website to get some ...
1
vote
2answers
111 views

High-level strategy for distinguishing a regular string from invalid JSON (ie. JSON-like string detection)

Disclaimer On Absence of Code: I have no code to post because I haven't started writing; was looking for more theoretical guidance as I doubt I'll have trouble coding it but am pretty befuddled on ...
4
votes
4answers
595 views

What is the responsibility or benefit of a Tokenizer?

Suppose I had a grammar like: object { members } members pair pair string : value value number string string " chars " chars char char chars number digit ...
0
votes
2answers
171 views

Lexer/Parser for multidimensional Languages

How does Lexer/Parser work in a 2D programming language like Funciton in order to transform such an unusual source-code to the correct AST?
0
votes
3answers
250 views

Do markup languages ​​have the equivalent concept of `semantics` that you can find in C or C++?

Maybe I miss something, but do the so called markup languages have the equivalent concept of semantics that you can find in C or C++? Judging from how you parse the language, you don't really have ...
2
votes
1answer
149 views

How and when should I design a simple mark-up language parser? [closed]

I want to write a simple markup language with its rendering engine. First, I am not completely sure when I should try this... I am only 12... But I am competent in C++ having learned through the Web ...
0
votes
2answers
106 views

Implementing a first basic interpreter: what should I learn first? [duplicate]

I'm about to implement my own very simple programming language, and an interpreter to execute code in that language. The language will be very basic. Example code: var x = 3 if x > 2 print x if x ...
0
votes
1answer
110 views

Traversing an AST using Visitors

I'm writing a compiler for a C-like language, and I'm looking for an elegant way to traverse my abstract syntax tree. I'm trying to implement the Visitor pattern, although I'm not convinced that I'm ...
0
votes
0answers
39 views

Dedupe while or after write

I have a summary tool written in python that reads input files, and writes them into a summary file. I have the following stipulations: No duplicates. If it exists, add a count to it. Is it ...
0
votes
2answers
73 views

Storing tokens during lexing stage

I am currently implementing a lexer that breaks XML files up into tokens, I'm considering ways of passing the tokens onto a parser to create a more useful data structure out of said tokens - my ...
-2
votes
1answer
135 views

What is the right way to parse HTML? [closed]

I've heard, that parsing HTML using the Cthulhu way is not very good. But what are the right ways to parse HTML? Or is it possible to parse it at all?
0
votes
0answers
52 views

Using JavaScript to find the correct offset in bundled files

I am currently making multiple parsers using PEGjs and have implemented my own partial preprocessor which using a RegExp finds and replaces '#include' directives with the desired files, resulting in a ...
6
votes
2answers
303 views

Slight extension for SQL prepared statements syntax. Need advice

In my database abstraction library I am extending SQL prepared statements syntax to hint a parser with expected literal type. I take it as a very essential improvement, my reasoning you can read here. ...
0
votes
1answer
81 views

Making a sldprt to PDB file converter?

I wanted to create a parser that can read a solidworks file and turn it into a protein data bank file. This has already been done in a program called DiamondCAD. ...
1
vote
1answer
119 views

About AST construction in LL1 non recursive parser

I have implemented a LL1 parser in a non recursive approach with a explicit stack. The following algorithm is from the Dragon Book: set zp to point to the first symbol of w; set X to the top stack ...
4
votes
1answer
154 views

How to line-break an email address?

After some discussion, I have come across a rather complicated situation. Say I intend to display an email address. I have, obviously, limited space available on the screen - be that browser or ...
2
votes
1answer
906 views

Using python to parse log files? [closed]

My first useful projects as a programmer has been python scripts that parse out relevant information from log files and do some analysis. I've bumped around and found my way to some functional ...
0
votes
0answers
169 views

Parsing Razor-style Templates

I want to build a template engine (ITT not another template engine...) based on Razor. I've been at it for quite a long time not getting anywhere and quite frankly I'm at my limit. I've tried rolling ...
-1
votes
1answer
80 views

Suggestion on how to fill a web form (several times) [closed]

I need to fill a form using data from a CSV file. I was planning to use CURL+PHP to do it, but then I realized the form has several steps (one on each page), plus it uses javascript to fill hidden ...
1
vote
2answers
167 views

Parser and interpreter knowledge as a way to gauge programmer ability [closed]

This is only anecdotal evidence but from my past encounters with programmers at various workplaces the programmers that understand the fundamentals of parsing and interpreting seem to be overall ...
6
votes
3answers
307 views

Clarification about Grammars , Lexers and Parsers

Background info (May Skip): I am working on a task we have been set at uni in which we have to design a grammar for a DSL we have been provided with. The grammar must be in BNF or EBNF. As well as ...
1
vote
0answers
45 views

Compound container for storing output from a parser

Consider a hierarchical structure like this directory [ [item_0;item_1;...;item_n] [item_0;item_1;inner_directory[...];item_m] other_directory [...] ] ...
1
vote
1answer
154 views

Grammar: start: (a b)? a c; Input: a d. Which error correct at position 2? 1. expected “b”, “c”. OR expected “c”

I am writing a parser (a parser generator for PEGs) and am trying to figure out how to report errors. Grammar: rule: (a b)? a c ; Input: a d Question: Which error message correct at position 2 ...
3
votes
1answer
169 views

Sentence Tree v/s Words List

I was recently tasked with building a Name Entity Recognizer as part of a project. The objective was to parse a given sentence and come up with all the possible combinations of the entities. One ...
0
votes
1answer
80 views

Should a structure be refactored into smaller structures?

In one of the projects (in C), I have the code that looks like this (the names are obfuscated specially for question to not give out details of the probject): typedef struct Module_s { /* These ...
5
votes
1answer
191 views

Does a GPL Bison grammar infect my application?

I am thinking about using a GPL Bison grammar for my own compiler. Will the grammar "infect" my parser such that it needs to be open source? The grammar - in terms of - the input of bison is GPL.
3
votes
1answer
213 views

Object oriented parser model review

First time I make use of Code Review. At the moment I'm creating an Android application and the company I'm developing it for, has a specific file format that can be used to give text based data ...
25
votes
1answer
981 views

The Inglish parser (for The Hobbit 1982)

Was fascinated to read about the text adventure game The Hobbit which featured an incredibly robust parser called "Inglish": ...Inglish allowed one to type advanced sentences such as "ask Gandalf ...
0
votes
1answer
184 views

Haskell: Best tools to validate textual input?

In Haskell, there are a few different options to "parsing text". I know of Alex & Happy, Parsec and Attoparsec. Probably some others. I'd like to put together a library where the user can input ...
2
votes
1answer
144 views

content fingerprint algorithm

I have a database containing just simple URLs. It is as simple as it sounds for now and URL can link to a website or a document(i.e. anything parseable to text). Now I have a simple code which ...
2
votes
2answers
2k views

An ideal way to decode JSON documents in C?

Assuming I have an API to consume that uses JSON as a data transmission method, what is an ideal way to decode the JSON returned by each API resource? For example, in Java I'd create a class for each ...
1
vote
0answers
133 views

Implementing semantic actions in table driven predictive parsing

When doing table driven predictive parsing on an LL(1) grammar, (as explained in good detail here for example), how can we augment the algorithm to allow processing of semantic actions while doing the ...
4
votes
2answers
262 views

Should I commit my generated parser to source control?

I'm using a parser generator to build a compiler. Should I commit the source files produced by the parser generator? I want to commit them to avoid a dependency on the parser generator during the ...
2
votes
4answers
209 views

The right place to embed a parser : Server or the Client?

I'm writing a database management software. It has a client-server architecture where the server has the piece of code to store and retrieve. The client has to get the input from the user, connects to ...
1
vote
2answers
288 views

How would my custom language be categorized?

I'm developing my own scripting language to solve some unique challenges for a project. The language takes source code and converts the contents into tokens, and then a command factory is used to ...
-3
votes
1answer
207 views

Best Object Oriented way of parsing a model with several fields(>50) and with null checks

I have a JSON-based data which contains many fields from a particular model. Any value from the list of fields can be null. I am trying to find the best object oriented way of parsing it and also it ...
4
votes
3answers
240 views

What is the proper way to distinguish between keywords and identifiers?

I'm aware that most modern languages use reserved words to prevent things like keywords from being used as identifiers. Reserved words aside, let's assume a language that allows keywords to be used ...
2
votes
1answer
872 views

Any ASN.1 parsers written completely in managed code & safe from buffer overflows

ASN.1 is a data format used on HTTPS certificates, and various other critical pieces of infrastructure. I need to analyze some potentially hostile ASN.1 data, and want to ensure my code is safe from ...