Analyzing (un)structured data to convert it into a structured, normalized format.

learn more… | top users | synonyms (1)

13
votes
2answers
1k views

What does scannerless parsing have to do with the “Dangling Else Problem”?

I do not understand this sentence from the Wikipedia article on the Dangling Else problem: [The Dangling Else problem] is a problem that often comes up in compiler construction, especially ...
9
votes
3answers
650 views

What the correct algorithm to invert italics in a mixed text?

The question motivations was depicted in the section below. There are many ways to make text italic, so, perhaps, there are more than one good "swap italics algorithm". The problem reveals some ...
32
votes
6answers
2k views

Why was strict parsing not chosen for HTML?

I am pretty much unaware about the detailed computing history that surrounds internet but it has always made me think why strict parsing was not chosen when creating HTML. Browsers accept any rubbish ...
3
votes
1answer
500 views

Chosing a parser for a code beautifier

I'm in the planning stage of making a code beautifier (similar to AStyle or Uncrustify) - originally I was going to just contribute to one of those projects, but reviewing their source led me to the ...
1
vote
4answers
616 views

Is it possible to create a single tokenizer to parse this?

This extends off this other Q&A thread, but is going into details that are out of scope from the original question. I am generating a parser that is to parse a context-sensitive grammar which can ...
-3
votes
2answers
299 views

What is the name of a grammar which can change its tokenizer in mid parse?

I was creating a language and discovered that my language tokenizer would have to change depending where in the parse it is. I.e. abc[1] would be parsed as 4 tokens (abc, [, 1, ]), where as { abc[1] ...
10
votes
3answers
1k views

Writing a Compiler Compiler - Insight on Use and Features

This is part of a series of questions which focuses on the sister project to the Abstraction Project, which aims to abstract the concepts used in language design in the form of a framework. The ...
1
vote
1answer
474 views

Should I use a formal grammar for my interpreted scripting language

I have a scripting engine I just published as an open source project. It's been sitting on my harddrive waiting for about a year. My engine of course isn't complete in any way, but it does work for ...
2
votes
1answer
609 views

How to choose a proper parser generator for PHP

Some programmers avoid regexes in some situations (see this popular @nickf comment), perhaps using a parsing framework such as Lex/Yacc. Others prefer to stay within PHP, perhaps using regular ...
15
votes
3answers
10k views

Implementing the Visitor Pattern for an Abstract Syntax Tree

I'm in the process of creating my own programming language, which I do for learning purposes. I already wrote the lexer and a recursive descent parser for a subset of my language (I currently support ...
1
vote
2answers
198 views

How are line/column position data dealt with in parser combinator libraries?

I'm building a parser using a parser combinator library. I need to keep track of where AST nodes started and ended in the textual input -- line and column numbers. How is this problem approached ...
10
votes
4answers
12k views

Generic file parser design in Java using the Strategy pattern

I am working on a product in which the responsibility of one of the modules is to parse XML files and dump the required content in a database. Even though the present requirement is only to parse XML ...
1
vote
2answers
164 views

Creating a text input simplifaction tool

I have been working on a project for several months now to write a web-based tool that will help me at work. I work at a call center and the CRM software that is used is wretched, we basically have to ...
2
votes
1answer
361 views

Practical reference for learning about graph reduction

Are there any practical references (with actual examples) for getting started implementing a small, lazy functional programming language with graph reduction? A reference that included the lexing and ...
8
votes
4answers
956 views

Using a “dead man's switch” to manage time-sensitive code

In our software environment, we often run a/b tests, as is probably good practice. However, our environment is set up such that, in very short order, the code starts to become very crufty with dead ...
3
votes
1answer
270 views

Showing a grammar is ambiguous

I have the following question taken from a compilers course exam: Show that the following grammar is ambiguous. S = XcY X = a Y = b | Z Z = bW W = d | ϵ I drew the following tree: Am I correct ...
0
votes
2answers
364 views

How to extract operators from the grammar productions for conflict resolution in LALR parser?

Is there some standardized or widely accepted algorithm for picking up operators in shift/reduce conflicts in LALR parser? The question is naive, my problem is not with implementing my solution, but ...
2
votes
1answer
580 views

REST efficiency on count and sum methods

For example, if I have a Customer Transactions Table and I create a REST for it. The fields are: date, description, and total_amount. I am trying to figure out which one is much more efficient when ...
7
votes
2answers
1k views

Language parsing to find important words

I'm looking for some input and theory on how to approach a lexical topic. Let's say I have a collection of strings, which may just be one sentence or potentially multiple sentences. I'd like to ...
8
votes
3answers
1k views

What is a real-world use case of using a Chomsky Type-I (context-sensitive) grammar

I have been having some fun lately exploring the development of language parsers in the context of how they fit into the Chomsky Hierarchy. What is a good real-world (ie not theoretical) example of a ...
1
vote
2answers
154 views

Extracting useful information from free text

We filter and analyse seats for events. Apparently writing a domain query language for the floor people isn't an option. I'm using C# 4.0 & .NET 4.0, and have relatively free reign to use ...
2
votes
1answer
1k views

How to add precedence to LALR parser like in YACC?

Please note, I am asking about writing LALR parser, not writing rules for LALR parser. What I need is... ...to mimic YACC precedence definitions. I don't know how it is implemented, and below I ...
0
votes
1answer
202 views

How lookaheads are propagated in “channel” method of building LALR parser?

The method is described in Dragon Book, however I read about it in ""Parsing Techniques" by D.Grune and C.J.H.Jacobs". I start from my understanding of building channels for NFA: channels are built ...
1
vote
1answer
334 views

How to read reduce/shift conflicts in LR(1) DFA?

I am reading an explanation (awesome "Parsing Techniques" by D.Grune and C.J.H.Jacobs; p.293 in the 2nd edition) and I moved forward from my last question: How to get lookahead symbol when ...
3
votes
2answers
4k views

How to get lookahead symbol when constructing LR(1) NFA for parser?

I am reading an explanation (awesome "Parsing Techniques" by D.Grune and C.J.H.Jacobs; p.292 in the 2nd edition) about how to construct an LR(1) parser, and I am at the stage of building the initial ...
3
votes
1answer
3k views

Extracting color profile information from JPEG files

I'm trying to look up info about reading JPEG's color profile info and to my surprise there's very little open specific how-to information on that regard, but rather lots of general explanation on ...
8
votes
4answers
298 views

How should I implement a command processing application?

I want to make a simple, proof-of-concept application (REPL) that takes a number and then processes commands on that number. Example: I start with 1. Then I write "add 2", it gives me 3. Then I ...
13
votes
5answers
474 views

How can I best manage making open source code releases from my company's confidential research code?

My company (let's call them Acme Technology) has a library of approximately one thousand source files that originally came from its Acme Labs research group, incubated in a development group for a ...
5
votes
2answers
680 views

Scripting custom drawing in Delphi application with IF/THEN/ELSE statements?

I'm building a Delphi application which displays a blueprint of a building, including doors, windows, wiring, lighting, outlets, switches, etc. I have implemented a very lightweight script of my own ...
3
votes
1answer
209 views

What does it mean that StAX parsers aren't complete or correct?

I've been browsing the net and found theses two slides on site of XOM (XML object model): slide1 slide2 Where they refer to SAX(push) and StAX(pull) parsers. So I'm wondering, what does it mean ...
18
votes
4answers
1k views

Generic rule parser for RPG board game rules - how to do it?

I want to build a generic rule parser for pen and paper style RPG systems. A rule can involve usually 1 to N entities 1 to N roles of a dice and calculating values based on multiple attributes of an ...
10
votes
2answers
1k views

Persisting natural language processing parsed data

I've recently started experimenting with natural language processing (NLP) using Stanford's CoreNLP, and I'm wondering what are some of the standard ways to store NLP parsed data for something like a ...
3
votes
3answers
677 views

design for a parser to handle very large files

I have written a program which records protocol messages between an application and a hardware device which matches each application request with each hardware response. This is so that I can later ...
1
vote
1answer
357 views

Does JAXP natively parse HTML?

So, I whip up a quick test case in Java 7 to grab a couple of elements from random URIs, and see if the built-in parsing stuff will do what I need. Here's the basic setup (with exception handling etc ...
11
votes
5answers
11k views

Can the csv format be defined by a regex?

A colleague and I have recently argued over whether a pure regex is capable of fully encapsulating the csv format, such that it is capable of parsing all files with any given escape char, quote char, ...
3
votes
3answers
799 views

How do I translate user input into a fictitious language?

For experimental reasons, I am trying to convert user input into a fictitious language. All of the translation can be 1:1. I would prefer if I could accomplish this with PHP. Should I use gettext ...
1
vote
1answer
151 views

How do I parse a header with two different version [ID3] avoiding code duplication?

I really hope you can give me some interesting viewpoints for my situation, because I am not satisfied with my current approach. I am writing an MP3 parser, starting with an ID3v2 parser. Right now ...
2
votes
2answers
160 views

parsing terminology: comments+whitespaces vs actual code

In languages like c/c++ spacing and comments are ignored and only actual code gets into compiler. I'm interested if there is accepted way of naming these two things? comments & spacing ...
5
votes
4answers
2k views

Programming Language Parser (in Java) - What would be a better design alternative for a special case?

Background I'm currently designing my own programming language as a research project. I have most of the grammar done and written down as context-free grammar, and it should be working as is. - Now ...
3
votes
1answer
229 views

How to support mixed grammars?

I am currently writing a grammar for a programming language (PowerBuilder) and I am also collecting documentation about parsing and compilers creation. Now I would like to add in my grammar (that ...
7
votes
2answers
349 views

Parsing multiple file formats/protocols

We are starting a project where we will need to write parsers for a bunch of binary file formats, each of them representing very similar data (time-value series from different measurement devices). ...
1
vote
2answers
1k views

How to create a Semantic Network like wordnet based on Wikipedia?

I am an undergraduate student and I have to create a Semantic Network based on Wikipedia. This Semantic Network would be similar to Wordnet(except for it is based on Wikipedia and is concerned with ...
8
votes
4answers
831 views

How are comments expressed in programming language grammars?

I'm learning how to build parsers using grammars, but I got stuck trying to express comments, because they can appear almost anywhere. This indicates that comments can be stripped from the token ...
0
votes
2answers
217 views

Using a parser to locate faulty code

Lately I've been working a lot in PHP and have run into an abnormally large number of parsing errors. I realize these are my own fault and a result of sloppy initial coding on my part, but it's ...
10
votes
5answers
30k views

Getting data from a webpage in a stable and efficient way

Recently I've learned that using a regex to parse the HTML of a website to get the data you need isn't the best course of action. So my question is simple: What then, is the best / most efficient and ...
4
votes
2answers
3k views

Is there a more modern program than lex or yacc, which does not require JVM?

Is there a "modern" bison and flex equivalent which does not require the JVM or .NET or similar "heavy" runtimes? Thanks for comments, edit: By modern, I mean for example Antlr and another one in ...
25
votes
1answer
11k views

C++11 includes std::stoi, why not std::itos?

I noticed to my glee that C++11 has a std::sto@ family of functions for easily unpacking ints/floats/longs whatever from strings. I'm surprised however, that the opposite isn't implemented. Why didn't ...
9
votes
6answers
4k views

Best way to parse a file

I'm trying to find a better solution for making a parser to some of the famous file formats out there such as: EDIFACT and TRADACOMS. If you aren't familiar with these standards then check out this ...
5
votes
3answers
2k views

What is this algorithm for converting strings into numbers called?

I've been doing some work in Parsec recently, and for my toy language I wanted multi-based fractional numbers to be expressible. After digging around in Parsec's source a bit, I found their ...
0
votes
1answer
178 views

What are known approaches to graphing algebraic expressions?

I am planning to build an expression parser that will be used to graph algebraic functions ( think TI-83 ) with JavaScript. Functions will take the form of f(x)= Aside from typical operators such ...