Analyzing (un)structured data to convert it into a structured, normalized format.

learn more… | top users | synonyms (1)

2
votes
3answers
213 views

Mapping different XML and CSV feeds

Not sure if this is the right venue to be asking this but here goes. A little background. I'm trying to build an ecommerce app that would allow sellers from other venues--like, amazon and newegg--to ...
5
votes
1answer
128 views

Alternative for eval() in javascript for expression evaluation

I'm looking at the alternative that can substitute the use of eval() and new Function() javascript techniques. I'm developing a solution build with javascript and the platform on which it is built (...
19
votes
12answers
7k views

How to write a command interpreter/parser?

Problem: Run commands in the form of a string. command example: /user/files/ list all; equivalent to: /user/files/ ls -la; another one: post tw fb "HOW DO YOU STOP THE TICKLE MONSTER?;" ...
6
votes
2answers
151 views

Do compilers utilize multithreading for faster compile times?

If I remember my compilers course correctly, the typical compiler has the following simplified outline: A lexical analyzer scans (or calls some scanning function) the source code character-by-...
0
votes
1answer
273 views

What is a lattice parser?

My friend asked me for help with his task: "Redeem a lattice parser with any programming language" - the problem is that he can't clearly explain me how the lattice parser should work. I've tried to ...
4
votes
5answers
209 views

How to store math expression in c++ list

I am parsing a infix math expression to a postfix form. I want to store it in a list like [4.5, 3, 0.25, +, -] so that I can process it once it's parsed. I could store it in a string again, but I ...
0
votes
0answers
35 views

Inheritance and Parcels

In Android development, we have Parcelable interface. Since I wish to build Data Models that will run on multiple apps and services (not just Android), I would like to add two jar libs that will share ...
3
votes
1answer
282 views

Sentence Tree vs. Words List

I was recently tasked with building a Name Entity Recognizer as part of a project. The objective was to parse a given sentence and come up with all the possible combinations of the entities. One ...
3
votes
1answer
45 views

Code style for parse-methods in Scala?

Say I have a class with multiple parameters: case class Foo(a: String, b: String, c: String, d: String, ... ) Now say I want to parse an instance of Foo from a Map[String, String]: // returns ...
1
vote
1answer
35 views

An elegant way to split text into words combined with adjacent punctuation and determine which punctuation mark it is

Firstly, I realize that question title is about as terrible as the sample code I'll post below, so please bear with me while I explain the problem more clearly, and if you have a better idea for the ...
4
votes
1answer
68 views

Right way to read from token list to implement parse tree

I'm trying to tackle the While Language problem on HackerRank using Scala. I've been given a set of grammar rules and have to implement the actual interpreter underlying it. I've decided to approach ...
15
votes
4answers
11k views

Implementing the Visitor Pattern for an Abstract Syntax Tree

I'm in the process of creating my own programming language, which I do for learning purposes. I already wrote the lexer and a recursive descent parser for a subset of my language (I currently support ...
0
votes
1answer
68 views

How do I create a parse tree, given a grammar? [closed]

I'm implementing a toy programming language and I've got as far as a working tokenizer that takes a stream of characters like this. 1 2 3 (2 *) map And outputs a stream of tokens { number "1" } { ...
1
vote
1answer
185 views

OOP: how to keep some state for a brief moment?

My job and my personal interests often have me writing parsing code, that is, code that converts from an input string written in a given formal language to a tree of model objects. I come from a C ...
1
vote
2answers
159 views

“Fuzzy” parsing in different languages

I have a bunch of plain-text like this: 1 MILE, PACE, PURSE $1,100. FILLIES & MARES N/W $541 L5 STARTS AE N/A $301 L5 & N/A $60 PS IN 2015-16 DRAW INSIDE ...
23
votes
3answers
2k views

In which process does syntax error occur? (tokenizing or parsing)

I'm trying to understand compilation and interpretation, step by step figuring out a total image. So I came up to a question while reading http://www.cs.man.ac.uk/~pjj/farrell/comp3.html this article ...
10
votes
4answers
13k views

Generic file parser design in Java using the Strategy pattern

I am working on a product in which the responsibility of one of the modules is to parse XML files and dump the required content in a database. Even though the present requirement is only to parse XML ...
24
votes
3answers
17k views

How exactly is an Abstract Syntax Tree created?

I think I understand the goal of an AST, and I've built a couple of tree structures before, but never an AST. I'm mostly confused because the nodes are text and not number, so I can't think of a nice ...
26
votes
1answer
12k views

C++11 includes std::stoi, why not std::itos?

I noticed to my glee that C++11 has a std::sto@ family of functions for easily unpacking ints/floats/longs whatever from strings. I'm surprised however, that the opposite isn't implemented. Why didn't ...
25
votes
8answers
4k views

Is it possible to statically predict when to deallocate memory---from source code only?

Memory (and resource locks) are returned to the OS at deterministic points during a program's execution. The control flow of a program by itself is enough to know where, for sure, a given resource can ...
4
votes
2answers
78 views

Testing strategies for interpreter language parser

For a recent personal project, I started working on an interpreter for my own programming language. One of the ground rules I set for myself on this project is that I need to properly test as much of ...
9
votes
2answers
218 views

language with two binary operators of same precedence, left-associative & right-associative

Are there any programming (or scripting) language (or some domain specific language) having two binary operators opl and opr of same precedence with opl being left-associative and opr being right-...
18
votes
4answers
1k views

Generic rule parser for RPG board game rules - how to do it?

I want to build a generic rule parser for pen and paper style RPG systems. A rule can involve usually 1 to N entities 1 to N roles of a dice and calculating values based on multiple attributes of an ...
0
votes
1answer
27 views

Is it possible to parse my grammar with multi-line productions without backtracking?

I'm playing around with creating a parser in PHP for my own flavor of BNF, to match strings against grammar in this BNF variant. It's still a work in progress and subject to change (I may even end up ...
2
votes
4answers
155 views

Is the var token necessary to signal variable declaration?

When creating a variable foo, Python lets you just write foo = bar. However many languages, like C# or JavaScript, require additional syntax like var foo = bar or foo := bar to signal the same thing. ...
0
votes
0answers
81 views

Parsing text with repeating blocks

I am parsing the responses from our Varnish load balancers in order to monitor the status of various nodes. One of the responses we get is text consisting of multiple blocks of data, one for each ...
5
votes
4answers
802 views

When to use ANTLR and when to use a parsing library

I've always wanted to learn how to write a compiler - I've decided to use ANTLR, and am currently reading through the book (its very good by the way) I'm pretty new to this, so go easy, but the jist ...
2
votes
3answers
6k views

An ideal way to decode JSON documents in C?

Assuming I have an API to consume that uses JSON as a data transmission method, what is an ideal way to decode the JSON returned by each API resource? For example, in Java I'd create a class for each ...
1
vote
1answer
45 views

How to handle user created expressions in application

In our application users can enter custom expressions to calculate certain things. For instance they can specify an invoice and define a number of lines for cost calculation. Example for a course ...
-1
votes
1answer
51 views

Tracking column (offset) in presence of tab characters

Imagine that you first write a compiler for your language where you necessarily report errors to the user. Compiler also collects location information for backend tools. They must know where the ...
2
votes
2answers
123 views

Regular syntax — what does it mean?

I am reading "Less is exponentially more" and there is a list of advantages of Go, first of them, quote "regular syntax (don't need a symbol table to parse)". What does it mean "regular syntax"? ...
1
vote
2answers
199 views

Declarative parser for JSON and DOM

I am parsing many JSONs and DOMs in PHP. The problem is that the my knowledge about the structure of the JSON/DOM is mixed with code. For example, suppose that I have few sources of first and last ...
4
votes
1answer
136 views

How to approach syntax errors when developing a lexical analyser?

I am writing an interpreted domain-specific language for my application. Each statement is parsed and executed as soon as lexical analyser decides that the end of statement is reached. It's handling ...
0
votes
1answer
141 views

Parsing conditional statements

I've written a small utility in Python3 to help me copy my music collection from my NAS to a mobile device. The usefulness of this is that it will auto-convert flac files to ogg-vorbis (to save space) ...
6
votes
1answer
189 views

quantitatively comparing AST shapes

How could one compare the shape of abstract syntax trees of similar source code programs (C, C++, Go, or anything compiled with GCC...)? I guess that plagiarism detection on source code would use ...
28
votes
4answers
3k views

How are comments usually parsed?

How are comments generally treated in programming languages and markup? I am writing a parser for some custom markup language and want to follow the principle of least surprise, so I'm trying to ...
0
votes
1answer
61 views

Is it a reasonable task to automatically convert ANTLR grammars to my own format?

I was working on my own parser generator tool in Clojure and then I realised I want to test it with some real programming languages. I can find a lot of existing language grammars only in antlr v4 ...
24
votes
6answers
2k views

What are the arguments against parsing the Cthulhu way?

I have been assigned the task of implementing a Domain Specific Language for a tool that may become quite important for the company. The language is simple but not trivial, it already allows nested ...
3
votes
6answers
351 views

In which layer I should implement file parsing?

in a simple multi layer architecture, in which layer do I have to implement something like parsing file. For example: I have a file and I have to extract specific information into an object. I think ...
0
votes
0answers
335 views

Writing Z80 table based assembler/disassembler

I have a long-term project: DIY computer with various processors. One of my wishes not only make hardware, but software too. So I started from assembler/disassembler for Linux, though there is a lot ...
8
votes
2answers
852 views

In layman's terms, what is left recursion?

According to one page on code.google.com, "left recursion" is defined as follows: Left recursion just refers to any recursive nonterminal that, when it produces a sentential form containing itself,...
5
votes
3answers
2k views

What is this algorithm for converting strings into numbers called?

I've been doing some work in Parsec recently, and for my toy language I wanted multi-based fractional numbers to be expressible. After digging around in Parsec's source a bit, I found their ...
6
votes
3answers
308 views

Is there such thing as an example driven parser generator or ad-hoc DSL development?

I'm intrested to know if there exists a tool that lets you input examples of valid documents and lets you then generalize from that to a reusable parser. I can imagine this, but everytime i start ...
5
votes
2answers
706 views

Scripting custom drawing in Delphi application with IF/THEN/ELSE statements?

I'm building a Delphi application which displays a blueprint of a building, including doors, windows, wiring, lighting, outlets, switches, etc. I have implemented a very lightweight script of my own ...
1
vote
1answer
437 views

How should I handle the fetching of cached data in iOS

I'm developing an app at work, this is my first big application and in my smaller projects I didn't use caching at all. What's currently happening When the user logs on for their very first time ...
1
vote
2answers
128 views

Is recursive-descent parsing a panacea for DoS threats posed by 'Evil' regexes? Or does evilness stem from the grammar?

ReDos attacks exploit characteristics of some (otherwise useful) regular expressions ... essentially causing an explosion of possible paths through the graph defined by the NFA. So does using a ...
2
votes
4answers
148 views

Clean Abstract Syntax Tree

I'm writing a toy compiler for fun. Basically, my problem is that I don't want to clutter the AST with stuff like debug information (symbol tokens, locations of tokens, etc) as well as data that the ...
5
votes
1answer
399 views

How should I test the HTML output my class creates?

As a learning project, I am trying to create something similar to the WebGrid that comes with ASP.NET MVC. Now this component MyGrid<T> looks like this: public class MyGrid<T> where T : ...
26
votes
5answers
3k views

Name for this type of parser, OR why it doesn't exist

Conventional parsers consume their entire input and produce a single parse tree. I'm looking for one that consumes a continuous stream and produces a parse forest [edit: see discussion in comments ...
2
votes
2answers
179 views

Incorporating functions into a Shunting-Yard algorithm implementation

tl;dr What would be a simple way of incorporating functions into a Shunting-Yard algorithm implementation? If only expressions like function(arg1, arg2, arg3) were allowed (where function is some ...