I have this image in my mind of a tropical island with a lone palm tree. A tribe of data modelers call the tree home. Occasionally, an app developer finds himself on the island looking for help. The tree-bound modelers yell out what the developer needs. The developer responds, “But how will this help me?” The modelers respond in unison, “Just listen to us and things will be fine.” The developer gets back in his canoe shaking his head as he disappears into the sunset.
This story illustrates an issue that I observe in the data architect community. Modelers with years of experience have developed a core set of beliefs of what a data architect does. When asked to be part of a project, they yell out what has to be done to satisfy the project’s data needs. Often this falls on deaf ears as developers fail to see the relevance of this work. The end result is disengagement of the data folk from the project team.
Sometimes we have to shake our tree and come down from our island to do our job. It is not about compromising our core data principles. It is about aligning our principles to current technologies and techniques. A data architect can quickly become disengaged if he/she does not work to keep their processes and practices in line with the app development world. Here are five tree shakers data architects must confront to remain relevant and engaged.
- You need a logical data model.
We understand the value of waterfall development, the importance of gathering business data requirements, and the need to build a database on a reliable foundation. Many of today’s development techniques are fast paced, iterative in approach and agile in their deployment. These are a challenge to the logical data modeler. I don’t advocate the dismissal of logical modeling. I do advocate starting my work with a logical model but delivering the data model my audience needs to do their job. I need a logical data model as does the enterprise. The developer does not. This is more commonly a physical data model that reflects the physical database. - You need a data model.
This is something that data architects need to take head-on when engaged in newer technologies such as No SQL and big data. These projects shake our tree vigorously. The developer runs the show. Questions are answered and relationships uncovered through coding and big data tools. This data is often unstructured and does not fit the relational model. Data architects can add value through their tools that are increasingly adding methods to reverse engineer these data structures and visually present them. The data architect is no longer the modeler of data but now the interpreter of data. - I have to model data.
Many a data architect determines their value by how many data models they produce. A larger portfolio equates to a more successful data architect. I see data architects as storytellers of data. You will find many of my posts talk about this subject. That is primarily why I abandoned the label of data modeler in favor of data architect. We do much more than model data. Part of telling that story is delivering metadata. It is helping others understand data with many tools. It is bridging the IT/business divide. Abandon measuring your value with data models as the currency. Measure your value by the methods you help others understand data. - I model the business and not the database.
I was taught this 25 years ago. My world was the pure business view free of technical limitations. I delivered that perfect logical data model. I let the DBAs have their way with it and implement a solution that worked in the database of choice. That scenario is not today’s reality. If you model data logically, you must provide a roundtrip view assuring the logical model, physical model, and database are in sync. If your world is limited by resources, you may need to deliver reverse engineered physical models from the databases with no logical model. Never forget that your job is to help people understand data. That customer may be developer, analyst, data scientist, DBA, manager, architect, accountant, and any other person in the enterprise. - Everyone can read a data model.
Another thing I was taught 25 years ago was that the data model graphically presents data in an easy to read format for all audiences. The ERD does provide a simpler snapshot of data. There are many people that use it and depend on it. There is a wider audience that just doesn’t get it. To this audience, I work hard to help them understand the model. Complex analytical models and large databases are difficult for even the most accomplished data model reader to comprehend. This is where our deployment of metadata, use of glossaries and data dictionaries, and availability to talk about this data becomes more important. Don’t make reading a data model a requirement of your customers. Observe and deliver this understanding in a way that meets their needs and experience.
Tom Bilcze
Data modeler tree shaker