168 reputation
1210
bio website None
location London, Brighton, UK
age
visits member for 2 years, 10 months
seen Jun 26 at 14:06

Mainly a SQL Developer, but also develop using:

C#, JavaScript, VB.NET, SSIS, SSRS, SSAS.


May
10
awarded  Fanatic
Mar
28
awarded  Informed
Feb
25
awarded  Caucus
Feb
25
awarded  Constituent
Jun
27
awarded  Critic
Jun
8
awarded  Caucus
May
16
awarded  Analytical
Mar
28
comment Designing a large database with multiple sources
Thankfully we do have a specialist and experienced OLAP developer.
Mar
28
comment Designing a large database with multiple sources
I thought of that, but would it noto be difficult to say normalise one source during the ETL, then add to those normalised tables during the next sources ETL, and the same for the next etc. It sounded like an odd way to go about it. Could be wrong of course!
Mar
28
comment Designing a large database with multiple sources
Primarily in the fact that all the data needs to be standardised I would say. But then the whole system has flaws of varying severities, lack of intelligent indexing, table structure, code etc. My thinking was to take a step back, and try and implement something extensible, with best practice in mind, rather than fixing and making do.
Mar
28
revised Designing a large database with multiple sources
edited title
Mar
28
comment Designing a large database with multiple sources
1) Yes, it does have to happen overnight. The other servers are production servers used for modelling and reporting and their DBAs get angry if we touch them during business hours. 2) There is currently only a PK on the large table. 3) This is certainly a possibility I have been thinking about, but I'm not sure if this is the most efficient way to let the data to consumed by the cubes and web app. 4) The servers are certainly powerful enough, I think its just the lack of best practice over the years causing inefficiencies.
Mar
28
awarded  Commentator
Mar
28
comment Designing a large database with multiple sources
Good spot! Ill leave it here for a while and see what happens, or bow the greater wisdom of the community and ask for it to be migrated.
Mar
28
revised Designing a large database with multiple sources
edited body
Mar
28
asked Designing a large database with multiple sources
Feb
27
awarded  Enthusiast
Oct
20
comment Are there best practices for checking for errors in business logic for DB refreshes?
Again thank you. Our Dev server(slowest) the overnight refresh takes 20 hours...(its done once a week at weekends) I couldn't do it locally at all, and the logic is very complicated! Equally, and here comes the rub, lets say its management that its pushing through these changes in the business logic, and not letting the dba's know. What ideally i want is to be able to produce some logic with which i can isolate issue before they become them, and if possible, be able to point to a respectable database management model... not even sure if this is exists too be honest.
Oct
20
comment Are there best practices for checking for errors in business logic for DB refreshes?
Is that something I can install locally and hook up to the servers? Seriously, the dba's need business cases for EVERYTHING. recently tried to get jdbc driver access and it wasn't pretty, even though relatively simple.
Oct
20
comment Are there best practices for checking for errors in business logic for DB refreshes?
Yes, its sadly very corporate. There has been Change Freezes imposed, which I understand to some extent, but it limits your ability to fix issues, and update logic based on say current political climate or legislation. You are definitely right about bringing up these issues, but its more the culture to get these changes through on the sly than to sort it properly. My thinking is to develop something independently that will highlight changes, and hopefully therefore a better way of going about things that can then be implemented a bit more solidly in the future.