2,010 reputation
637
bio website
location
age
visits member for 2 years
seen 5 hours ago

Learn PHP the right way: http://www.phptherightway.com

twitter: @songoko20000


Apr
13
awarded  Popular Question
Apr
12
awarded  Popular Question
Apr
12
revised Choosing an ubiquitous language across different bounded contexts
added 665 characters in body
Apr
5
awarded  Nice Question
Apr
4
comment Validating objects with each other - Design Pattern needed
@Berlo I think you can use the specification pattern
Apr
1
comment how and should I 'unit test' an entire package?
You don't! That's why we have integration/functional/acceptance tests.
Apr
1
comment Is Object Oriented Design necessary when building Symfony web apps?
Frameworks won't write ur business logic.
Mar
27
answered Should static data be stored in a database or somewhere else?
Mar
24
awarded  Yearling
Mar
22
comment Is there a name for the number of values a variable can take?
format your question plz :)
Mar
19
comment Processing csv files (alternative to business logic in DB)
@MichaelT Yup using PHP also affected other normal users doing day to day tasks (Customer service, Operations,...etc). Processing in PHP even in batches took a long time and a huge memory specially since several files get processed simultaneously.
Mar
19
comment Processing csv files (alternative to business logic in DB)
@MichaelT well the validation rules are many and change regularly. Sometimes it varies among different suppliers. We were hoping to encapsulate the rules into specifications to proper test them and change them easily. I don't know the alternative to not instantiate them all. Are you implying that I should create a unit of work maybe? like processing 100 lines then saving them then another batch?
Mar
18
comment Processing csv files (alternative to business logic in DB)
@MichaelT We are currently loading the file into the dummy table. This works fine for apache/php performance as the whole load is on the database itself to process the data. When we parse the file first using PHP to create our entities we run out of memory as the number of created object in enormous.
Mar
18
comment Processing csv files (alternative to business logic in DB)
@RobertHarvey Thanks for the ETL link. It seems we were doing just that without knowing!
Mar
18
comment Processing csv files (alternative to business logic in DB)
+1 I never thought about splitting the process into 2 databases. Gonna talk with our database admin and see if that is feasible.
Mar
18
comment Processing csv files (alternative to business logic in DB)
@DocBrown I updated the question with some numbers. We approximately receive 90 files per day and each file contains about 90 thousand records. Sometimes 200 records in during sales and Christmas. We have mainly 2 tables 2 fill in addition to 2 more for invalid orders and history requirements. We have very complex validation rules on the data. We need to combine elated records to common aggregates then validate again. Believe me it's a nightmare for maintenance. A change to the validation or any new requirement introduces numerous hard to detect bugs. I was hoping to use unit test, but in vain
Mar
18
comment Processing csv files (alternative to business logic in DB)
@MichaelT please check the update.
Mar
18
revised Processing csv files (alternative to business logic in DB)
added 385 characters in body
Mar
18
asked Processing csv files (alternative to business logic in DB)
Mar
17
comment Reading a file line by line for large files efficiently in PHP?
y don't u use The SplFileObject class‌​?