Analyzing (un)structured data to convert it into a structured, normalized format.

learn more… | top users | synonyms (1)

2
votes
1answer
557 views

Idea of an algorithm to detect a website's navigation structure?

Currently I am in the process of developing an importer of any existing, arbitrary (static) HTML website into the upcoming release of our CMS. While the downloading the files is solved successfully, ...
2
votes
2answers
240 views

“Hand written” recursive descent parser with “catch all” rule

I'm trying to write a (scannerless) recursive descent parser with a "catch all" rule for the following "Mustache template" grammar (simplified here): content : (variable_tag | section_tag | ...
2
votes
1answer
224 views

content fingerprint algorithm

I have a database containing just simple URLs. It is as simple as it sounds for now and URL can link to a website or a document(i.e. anything parseable to text). Now I have a simple code which ...
2
votes
1answer
391 views

Practical reference for learning about graph reduction

Are there any practical references (with actual examples) for getting started implementing a small, lazy functional programming language with graph reduction? A reference that included the lexing and ...
2
votes
1answer
297 views

Best way to parse optional grammar rules?

I started to write a hand written parser just for fun. For simple rules it is working, but I have problem with the optional grammar rules. I marked the optional rules with a question mark. Here is a ...
2
votes
2answers
178 views

Incorporating functions into a Shunting-Yard algorithm implementation

tl;dr What would be a simple way of incorporating functions into a Shunting-Yard algorithm implementation? If only expressions like function(arg1, arg2, arg3) were allowed (where function is some ...
2
votes
2answers
150 views

Algorithm to go from infix notation to a tree

I've been trying to figure out an algorithm to go from an infix equation to a syntax tree, like so: (1+3)*4+5 + * 5 + 4 1 3 However, I don't just want it to handle operators, I ...
2
votes
1answer
183 views

Writing a parser on top of an XML-based AST: am i doing it right?

I have a sort of AST defined in XML that i'm trying to parse and evaluate. The XML tree contains the tokens and all the information i need. However, i'm finding it difficult to do it "properly". Here'...
2
votes
2answers
727 views

Traversing an AST using Visitors

I'm writing a compiler for a C-like language, and I'm looking for an elegant way to traverse my abstract syntax tree. I'm trying to implement the Visitor pattern, although I'm not convinced that I'm ...
2
votes
1answer
629 views

How to choose a proper parser generator for PHP

Some programmers avoid regexes in some situations (see this popular @nickf comment), perhaps using a parsing framework such as Lex/Yacc. Others prefer to stay within PHP, perhaps using regular ...
2
votes
2answers
5k views

Fastest C++ XML parsing library

I have thousands of .xml files from size 1MB-45MB (no DTDs). I need to parse and further manipulate these XML files before generating separate .xml files with the results of my regex. What the ...
2
votes
4answers
148 views

Clean Abstract Syntax Tree

I'm writing a toy compiler for fun. Basically, my problem is that I don't want to clutter the AST with stuff like debug information (symbol tokens, locations of tokens, etc) as well as data that the ...
2
votes
2answers
418 views

Building a string parser for user command and control?

My goal is to build a command parser that has basic syntax and multiple possible branches at each point. These commands come from users of the system and are text input (no GUI). The basic syntax is ...
2
votes
3answers
213 views

Mapping different XML and CSV feeds

Not sure if this is the right venue to be asking this but here goes. A little background. I'm trying to build an ecommerce app that would allow sellers from other venues--like, amazon and newegg--to ...
2
votes
0answers
70 views

Truncating HTML content at specific content blocks

I have HTML content in my DB and I would like to present a list of these individual items but truncate each of them so they're not fully displayed. I would like to keep truncated items human-...
2
votes
0answers
50 views

Argument over performance using Convert.ChangeType in Web Applications [duplicate]

A debate has been going on at work about using Convert.ChangeType. A couple of fundamental assumptions to this discussion are delineated below: 1. The discussion is within the context of web ...
2
votes
0answers
70 views

Loop Unfolding and Named Significant Bits

I've been writing a Parser Compiler for the last seven or so years, and I recently got to the point (yet again, never satisfied) of structuring the portion dealing with the portions of the language ...
2
votes
1answer
511 views

How and when should I design a simple mark-up language parser? [closed]

I want to write a simple markup language with its rendering engine. First, I am not completely sure when I should try this... I am only 12... But I am competent in C++ having learned through the Web ...
2
votes
0answers
209 views

Implementing semantic actions in table driven predictive parsing

When doing table driven predictive parsing on an LL(1) grammar, (as explained in good detail here for example), how can we augment the algorithm to allow processing of semantic actions while doing the ...
1
vote
5answers
1k views

I need to develop a parser. Can I use Lex and Yacc for the purpose?

I need to extract very particular data from log files(of different types and formats). Since I am a recent college passout; my mind ran to using Lex and Yacc for the purpose. Now I have the following ...
1
vote
3answers
292 views

Do markup languages ​​have the equivalent concept of `semantics` that you can find in C or C++?

Maybe I miss something, but do the so called markup languages have the equivalent concept of semantics that you can find in C or C++? Judging from how you parse the language, you don't really have ...
1
vote
3answers
618 views

Why Double.parseDouble(“ABC”) not returns Double.NaN?

This code: Double.parseDouble("ABC") throws a NumberFormatException. Why is it wrong to expect a Double.NaN (NaN is literally Not-A-Number). A working example is this: public static void main(...
1
vote
2answers
261 views

Parser and interpreter knowledge as a way to gauge programmer ability [closed]

This is only anecdotal evidence but from my past encounters with programmers at various workplaces the programmers that understand the fundamentals of parsing and interpreting seem to be overall ...
1
vote
1answer
211 views

Grammar: start: (a b)? a c; Input: a d. Which error correct at position 2? 1. expected “b”, “c”. OR expected “c”

I am writing a parser (a parser generator for PEGs) and am trying to figure out how to report errors. Grammar: rule: (a b)? a c ; Input: a d Question: Which error message correct at position 2 ...
1
vote
2answers
131 views

Language/Programming term for paired delimiters [closed]

Can someone help me find the language and/or programming term for delimiters (?) that must be paired? Quotes, parenthesis, angle-brackets, square-brackets, etc. are often used to symbolize these ...
1
vote
1answer
164 views

Is it bad to implement a language in other two languages? [closed]

Ok, so I have some understanding about parsers and compilers, at least the basics of how it works, and i've written a calculator and a really small toy language that compiles to another high-level ...
1
vote
2answers
90 views

Passing context around AST nodes

I have various objects inside my AST, such as IfBlock, FunctionBlock, LogicExpression, etc. All of those objects share a context, which is basically a hashmap with some variables. It's a very simple ...
1
vote
4answers
663 views

Is it possible to create a single tokenizer to parse this?

This extends off this other Q&A thread, but is going into details that are out of scope from the original question. I am generating a parser that is to parse a context-sensitive grammar which can ...
1
vote
2answers
1k views

How to create a Semantic Network like wordnet based on Wikipedia?

I am an undergraduate student and I have to create a Semantic Network based on Wikipedia. This Semantic Network would be similar to Wordnet(except for it is based on Wikipedia and is concerned with "...
1
vote
2answers
128 views

Is recursive-descent parsing a panacea for DoS threats posed by 'Evil' regexes? Or does evilness stem from the grammar?

ReDos attacks exploit characteristics of some (otherwise useful) regular expressions ... essentially causing an explosion of possible paths through the graph defined by the NFA. So does using a ...
1
vote
2answers
418 views

High-level strategy for distinguishing a regular string from invalid JSON (ie. JSON-like string detection)

Disclaimer On Absence of Code: I have no code to post because I haven't started writing; was looking for more theoretical guidance as I doubt I'll have trouble coding it but am pretty befuddled on ...
1
vote
1answer
371 views

Haskell: Best tools to validate textual input?

In Haskell, there are a few different options to "parsing text". I know of Alex & Happy, Parsec and Attoparsec. Probably some others. I'd like to put together a library where the user can input ...
1
vote
1answer
2k views

Why nginx's http parser doesnt use regular expressions?

I see the http parser written by Igor Sysoev for nginx does not use regular expressions https://github.com/joyent/http-parser What could be the main reason for such design decision? I guess I could ...
1
vote
1answer
35 views

An elegant way to split text into words combined with adjacent punctuation and determine which punctuation mark it is

Firstly, I realize that question title is about as terrible as the sample code I'll post below, so please bear with me while I explain the problem more clearly, and if you have a better idea for the ...
1
vote
1answer
185 views

OOP: how to keep some state for a brief moment?

My job and my personal interests often have me writing parsing code, that is, code that converts from an input string written in a given formal language to a tree of model objects. I come from a C ...
1
vote
1answer
124 views

File validation rules

I have an application that can accept CSV files to run some operations. The files look like: CREATE USER:username,last_name,first_name,age user1,Smith,John,23 user2,Poppins,Mary,257 There are a ...
1
vote
1answer
97 views

Run a c++ program under lots of different data maps

I want to run a c++ program to process a lot of data from different xml files and output results. I run the program once per file and potentially have around 50 different files. The trouble is each ...
1
vote
3answers
1k views

Parsing mathematical expressions with two values that have parentheses and minus signs

I'm trying to parse equations like the ones below, which only have two values or the square root of a certain value, from a text file: 100+100 -100-100 -(100)+(-100) sqrt(100) by the minus signs,...
1
vote
1answer
533 views

PHP Simplest XML tokenizer for string processing?

I need to do some text processing tasks with short XML fragments. The section "details" shows an example... My solution is to use a tokenizer based on regular expresions, but it is not elegant and not ...
1
vote
1answer
495 views

Should I use a formal grammar for my interpreted scripting language

I have a scripting engine I just published as an open source project. It's been sitting on my harddrive waiting for about a year. My engine of course isn't complete in any way, but it does work for ...
1
vote
2answers
208 views

How are line/column position data dealt with in parser combinator libraries?

I'm building a parser using a parser combinator library. I need to keep track of where AST nodes started and ended in the textual input -- line and column numbers. How is this problem approached and/...
1
vote
2answers
159 views

“Fuzzy” parsing in different languages

I have a bunch of plain-text like this: 1 MILE, PACE, PURSE $1,100. FILLIES & MARES N/W $541 L5 STARTS AE N/A $301 L5 & N/A $60 PS IN 2015-16 DRAW INSIDE ...
1
vote
1answer
45 views

How to handle user created expressions in application

In our application users can enter custom expressions to calculate certain things. For instance they can specify an invoice and define a number of lines for cost calculation. Example for a course ...
1
vote
2answers
198 views

Declarative parser for JSON and DOM

I am parsing many JSONs and DOMs in PHP. The problem is that the my knowledge about the structure of the JSON/DOM is mixed with code. For example, suppose that I have few sources of first and last ...
1
vote
1answer
435 views

How should I handle the fetching of cached data in iOS

I'm developing an app at work, this is my first big application and in my smaller projects I didn't use caching at all. What's currently happening When the user logs on for their very first time ...
1
vote
1answer
57 views

Can table of content be parsed using some formal grammar?

A table of content can look like: Preface Table of Content Chapter 1 ... 1.1 ... 1.1.1 ... 1.1.2 .... 1.2 ... Summary Exercises Chapter 2 ... ... Appendix ... A ... A.1 ... A.2 ... B ... References ...
1
vote
1answer
491 views

About AST construction in LL1 non recursive parser

I have implemented a LL1 parser in a non recursive approach with a explicit stack. The following algorithm is from the Dragon Book: set zp to point to the first symbol of w; set X to the top stack ...
1
vote
2answers
294 views

How would my custom language be categorized?

I'm developing my own scripting language to solve some unique challenges for a project. The language takes source code and converts the contents into tokens, and then a command factory is used to ...
1
vote
2answers
165 views

Creating a text input simplifaction tool

I have been working on a project for several months now to write a web-based tool that will help me at work. I work at a call center and the CRM software that is used is wretched, we basically have to ...
1
vote
2answers
158 views

Extracting useful information from free text

We filter and analyse seats for events. Apparently writing a domain query language for the floor people isn't an option. I'm using C# 4.0 & .NET 4.0, and have relatively free reign to use ...