Reputation
Top tag
Next privilege 200 Rep.
See reduced ads
Badges
4
Newest
 Editor
Impact
~658 people reached

  • 0 posts edited
  • 0 helpful flags
  • 4 votes cast
Feb
4
comment Should we reuse web app logs for user behaviour analytics?
@Llopis, thanks, I'm aware of that. Which is why I was wondering how should the integration with the existing logging system should be done; while not writing GA tracking events all over the app.
Feb
4
comment Should we reuse web app logs for user behaviour analytics?
Thanks for the answer. Where do services like Google Analytics fit into this? Where such consumers be defined? Frontend? Backend?
Feb
4
asked Should we reuse web app logs for user behaviour analytics?
Aug
11
comment Data generation system modular design
@Matt The technology is basically open for discussion. Although, since the question was asked, we settled on Python. Currently, we intend to implement each module as a Python module which exposes a 'run' method. All modules import a shared configuration module, which is initialized by a controller script. Other than initializing the config module, the controller also runs, according to the mentioned order, all other modules. Things like managing DB connection, loggers, etc. are also done by the controller. Any feedback and further advice will be much appreciated.
Aug
7
comment Data generation system modular design
please see my second edit to the question.
Aug
7
revised Data generation system modular design
added 310 characters in body
Aug
7
comment Data generation system modular design
Yes, modules will generally consume different data (some may be run parallely on the same input data). Yes, the structure of data will change between modules, as modules in one step, consume data generated by a module in the previous step.
Aug
7
revised Data generation system modular design
deleted 2 characters in body
Aug
7
comment Data generation system modular design
Thanks for the answer. I have a few concerns. Each module needs the complete data from the previous step. So they cannot be piped. I would need to wait for a module to complete its job before running the next step. Additionally, can you please give some ideas of how should modules get the data (if not from a DB)? What are some reasonable ways to transfer data (a lot of it) between modules? I can only think of a DB for this. Also, what are some ways of building a module to receive data instead of pulling it (assuming data can't be piped)?
Aug
7
awarded  Editor
Aug
7
revised Data generation system modular design
added 286 characters in body
Aug
7
asked Data generation system modular design
Jul
30
awarded  Scholar
Jul
30
accepted How should data transfer between a client and a web API for normalized data be designed?
Jul
29
comment How should data transfer between a client and a web API for normalized data be designed?
About the number HTTP requests in options 1, let's say the API can provide datums as a batch. You give it a bunch od record IDs all at once. In this case, the number of HTTP requests can be reduced if the client asks for records as a batch in one call.
Jul
29
comment How should data transfer between a client and a web API for normalized data be designed?
Thanks for the detailed answer! You have good inputs. About coupling between API and data - can you really make a generic API? Should you? I mean, as I see it, the API should follow the structure of your data; otherwise you are unnecessarily designing two different data schemas, one for the DB and one for the API output.
Jul
29
awarded  Student
Jul
29
asked How should data transfer between a client and a web API for normalized data be designed?
Nov
16
awarded  Supporter