Join Tag1 Team Talks for the launch of our detailed blog series dedicated to guiding users through the Drupal 7 to Drupal 10 Data Migration process. Created by migration expert Mauricio Dinarte, our series caters to developers and site owners alike by covering theory, practical examples, and a breadth of topics ranging from data conflicts and content model changes to debugging migrations.

The series is a step-by-step guide addressing complex and costly aspects of the data migration process, supported by a local DDEV environment setup. Furthermore, it addresses common challenges and provides insights into lesser-known options and techniques for a successful Drupal data migration.

[00:00:30] Joining me on the show today is Tag1 Team member, Mauricio Dinarte, a well known Drupal migration expert and the author of this series. We're building a guide that's going to help everyone who needs to transition from Drupal 7 to the latest version of Drupal, whether you're a developer looking to learn hands on how to upgrade your site or a site owner embarking on a large scale migration with an agency partner, and you want to better understand the most complex and costly aspect of the migration process.

[00:01:19] We're the largest concentration of top Drupal contributors and security team members at any organization. And we're responsible for a lot of the innovations that have helped make Drupal the success that it is, including the creators and maintainers of Drupal's Migrate API, which is the tool that's used to migrate your data in Drupal.

[00:02:03] and more. And he's published many articles as well as another 30 part migration series back in 2019. Mauricio, thank you so much for joining me and welcome back to the show.

[00:02:15] Michael Meyers: Before we get started, I just want to say thank you so much for doing this. Um, when I read your, your first post, and I think people are going to think this when they do as well.

[00:02:48] So, uh, for example, I think, you know, one of the things that, uh, surprised me is that the Feeds module is something that people use, uh, to do migrations instead of the Migrate API. I've been doing Drupal for 20 [00:03:00] years. I've done a lot of complex migrations and I didn't know that that was a thing. So I'm really confident that whoever is going to be using your series, you know, developers, business folks, uh, you know, it's going to be an amazing resource to help them plan and work on their migration.

[00:03:25] Mauricio Dinarte: Uh, sure. Uh, the series is what I like to call a very opinionated guide. And by that, I mean, um, there are multiple ways in which you can tackle a migration, but I will show one very specific way.

[00:04:16] So I try to give you as much information. So you will make an informed decision and hopefully you are convinced to use the Migrate API. And when you do so. Then we go into the example, which, uh, includes migrating configuration and migrating content, multiple entities of each type and general, uh, like gotchas that you might, um, encounter.

[00:05:11] In addition to that, we go through different examples, not only About how to migrate the data itself, but how to accommodate content model changes. Let's say that you had something that used to be a node, but now you want to, uh, use user entities or you want to leverage layout builder that was not being used before.

[00:06:02] Um, how to create custom source or process plugins to you know, accommodate different transformations that may be needed. And at the end, we're going to talk about events and hooks and how to, um, debug migrations, which is something that I do every day. Uh, every time that I work with a migration, there will be something that doesn't work as expected.

[00:06:35] Michael Meyers: Yeah. You really do cover a lot of bases, you know, even within a single aspect, you know, I mentioned Feeds earlier. Uh, another thing that I didn't think about is manual migrations, right? For a small site, it may not make sense to use any API or, uh, or Feeds module, or maybe for even a small aspect of really large sites.

[00:07:26] Um, at the top of the show, I mentioned that I think everybody can learn something from this series, uh, but it is without a doubt, you know, a step by step guide to how to, so the primary audience is developers. Um, what level of expertise do I, if I want to follow this along hands on. And use this to, you know, really learn how to do a migration.

[00:07:49] Mauricio Dinarte: Um, ideally you have some familiarity with the Migrate API. I would like to clarify that this series is focused on Migrating from Drupal 7 to [00:08:00] Drupal 10. But this is just one use case of the Migrate API, as I covered in my 2019 series, you can use, um, you know, the same core API to migrate from, um, like hard coded data or Excel or Google Sheet or JSON or XML.

[00:08:37] Again, I'm going to mention it, but it's not going to be the primary focus of this. This is going to be. Um, you know, as much as possible hands on example of going from going from Drupal 7 to Drupal 10. So as long as you are familiar with that, uh, you should be good to go. And one of the things that I find really empowering about a Migrate API in Drupal 10 is that you need very little [00:09:00] programming knowledge.

[00:09:33] And when we talk about debugging, but most of the series, you will be able to follow along. And yes, it is, uh, you know, written with developers in mind. But I want to point out a few things. One, I really like to distill content so that anybody can follow it. And by that, I mean, literally anybody, um, if you are a business owner, if you are a small shop, if you are a one person organization that [00:10:00] is doing this type of work, um, there will be something for you.

[00:10:28] But, um, in general, if you are curious about migrations, if you want to learn how to do it, um, as long as some familiarity of site building concepts and the Migrate API, uh, that suffices to follow along.

[00:10:46] There's some awesome background material, uh, that can really help people get the most out of this guide. So if you're going to follow this guide, please make sure to check that out as well.

[00:11:12] Mauricio Dinarte: Um, we are going to provide, uh, an example Drupal 7 site, uh, and this comes with a DDEV, uh, installation.

[00:11:42] And we also provide a Drupal 10 environment. Uh, we did it and this one we divide, we have like the final version with the full complete example that you can use as a reference, but using Git and Git branches, we also like have [00:12:00] like, uh, checkpoints that, you know, if you are starting from scratch and you follow the series, uh, like one article at a time, you can use this branch to like, start from scratch.

[00:12:29] You, you get the full, the full thing ready.

[00:12:56] Mauricio Dinarte: Yes. Well, that, that was too much and I learned my lessons. [00:13:00] So, um, you know, back then I basically have full sponsorship for a month. To work only on the series. And even with that, it was tough. So the plan now is to write two articles per week. I think there's going to be a varied cadence in my case to make it sustainable, but also for the reader to be able to follow along.

[00:14:02] It is shockingly easy to use. Um, is the assumption, you know, because it's so easy that anybody can just get up and running with it. Um, you know, are you going to provide step by step setup, installation instructions, you know, what comes along with this environment?

[00:14:26] Like for each Drupal 7 and then for Drupal 10, about 10 commands that you should run. And with that, you get everything. I really enjoy DDEV. I've been using DDEV exclusively for the last like three years at least, and it makes developing and debugging and performance testing for Drupal and for other projects for that matter, uh, very easily.

[00:15:16] So if you have a different local environment of choice, you can import that into your environment and follow along, or if you want to just like practice this on your own website, you can follow the same techniques, uh, and do it with your, with your real project. Uh, but ultimately you are going to have all the tools and all the resources.

[00:15:43] Michael Meyers: Dive in and migrate your site with the series. Love it. In the introduction post that I mentioned that I really loved, you take a step back and you discuss, you know, the different components, what makes up a migration, how they factor into it.

[00:16:14] Mauricio Dinarte: Um, there is a saying that I really like in English. It says, uh, measure twice, cut once. When I present on migrations, I usually say measure twice, cut once.

[00:16:52] Uh, so with that in mind, um, I, you know, I take a step back and review the Drupal 7 site [00:17:00] is from a content model perspective. Uh, that includes reviewing all the Content types, how many. Um, Nodes for content types there are? How many of those are published or not. Uh, we have seen sites that, you know, there are 10 years old and a lot of growth has accumulated that people are not interested in moving over.

[00:17:45] So again, like this is more like an x ray kind of thing. Uh, you review the data and make, uh, you know, some analysis on top of that. In addition to the Nodes, we also review Field types, like what field types [00:18:00] are in use. Uh, we review Vocabularies, we review Files, file types, um, if you're going to be using media, uh, we, we consider like local files, remote files, uh, we review, um, if you're using Paragraph, if you're using organic groups, if you're using rules, basically, um, anything that has the possibility to impact the content model.

[00:18:59] [00:19:00] And in one of the articles, we're actually going to use the Drupal 7 site that we have and give you the SQL queries that you can run against the database to fill out, uh, the template.

[00:19:31] And your series dives into these, uh, but I want to talk a little bit more about the configuration and data part, because, you know, when you think about D7 to D10, and how much changes, like, I wonder, like, is it really worthwhile or possible to migrate configuration from 7 to 10? Um, can you talk a little bit more about, you know, what you have lined up on that front?

[00:20:21] For example, Media in core became available in Drupal 8 and by extension it is also available in Drupal 10. It hasn't been moved out of core yet. Layout builder didn't exist before. You might want to use layout builder. So for one, Um, if you're going to migrate the configuration, which includes content types, uh, which includes, uh, media entities, which includes, uh, vocabularies, uh, organic groups and commerce stores, a lot of other things, if you're going to make a one to one copy, uh, the Migrate API will, for the most part, work out of the [00:21:00] box.

[00:21:20] So even though they are very similar in name. The underlying structure of the entities and the tables is totally different. So you need to be able to accommodate, uh, you know, the data transformations to, uh, from, from one version to another. Um, but, but again, like in our example, we're going to show how to automatically migrate the content types and fields, uh, view modes.

[00:22:22] Um, I have used it to automate, you know, Views migrations and even though it may not get you all the way there, it is a good starting point. So, you know, Views is also part of configuration that we're going to be demonstrating how to migrate.

[00:22:41] So migrations involve a lot of complex transformations, potentially, you know, the How-to guide that you're writing is based on the Migrate API, which is by far the most popular approach to doing data migrations. And it's a super powerful and flexible tool that enables you to do all of these things, right?

[00:23:11] Mauricio Dinarte: Yes, as you said, the Migrate API is very big and it will be unfeasible to write everything in 31 days.

[00:23:39] So there will be at least one throughout the, throughout the example. Uh, we're also going to go over hooks and events, um, how you can react when these are fired as part of the migration process and the different phases of the migration process, so that you can either alter the data, [00:24:00] or Or like that you are receiving from, um, Drupal 7 or alter the data right before it is being sent to Drupal 10 for storage.

[00:24:37] And one of those things, if you wanted, is, uh, enable validation. So, um, validations are sometimes tricky when it comes to migrations. They are very important to, um, to catch, uh, potential data inconsistencies. Uh, but sometimes you get a very obscure error and you just like don't know what to [00:25:00] do about it. So again, like we're going to be distilling some of the knowledge that we have gained over the years.

[00:25:34] And that is a flag in the, in the file entity. So even though I was migrating media entities at this point, my error originated from a previous migration in file. So that, that's not very common that you. have, uh, like validation programs in two different migrations. Usually it will be on only on the one that you're working on.

[00:26:15] So, um, again, not to go too deep into that now, but, uh, we're going to show how to write custom source plugins. Uh, we're going to show how to write custom process plugins. And in the case of the destination plugins, it's very rare that you need to write one. Um, so we're just going to mention it. But the example itself is not going to include one destination plugin.

[00:26:55] Michael Meyers: As well as how to debug and, you know, look into challenges that might arise [00:27:00] from doing all of that.

[00:27:13] Mauricio Dinarte: Um, yeah, I would say that debugging, um, debugging, some people think it's complicated. Uh, some people just don't want to try it. And to be honest, after using a proper debugger, I cannot go back, like you get access to everything, like you can access, um, the state of the application, like what variables are available at this stage of the migration and what comes later, like how the transformations go, even if you want, like one line at a time, what is happening under the hood.

[00:28:20] And other topics will also be included like tips and how-tos some, you know, even performance considerations. So if you are in the. articles of migrating files. There will be considerations about debugging, about performance that pertain to files migrations.

[00:29:00] Michael Meyers: In you know, 30 plus posts, you're going to cover a lot of topics, pretty much everything that the average site owner means and more, uh, but obviously you can't cover everything, right? You mentioned, you're going to go into views and limited depth and detail. Uh, the migrant API is so massive. You could write a book on it.

[00:29:27] Mauricio Dinarte: Yes. Um, I will cover most of what Drupal Core provides and a handful of contributed models. That being said, things that I will not be covering are organic growth migrations.

[00:30:02] So there is not an easy way to abstract that, um, at least in the context of the example that we are providing. So, uh, organic groups to groups that is not being covered. Something else is rules, uh, rules, and there are Module and that you can get similar functionality. EECA is a very common detailed alternative, but it's the, there is no automatic upgrade path in between the two.

[00:30:55] Um, sometimes they use rules to automatically publish or [00:31:00] unpublish content. There are models that can do that without using rules. There are rules for sending an email. You can do that again, either via models or via custom code. So. That's why an audit is very important because it, it gives you like a better idea of, um, do I need to keep this module?

[00:31:48] So even though the site is not multilingual in itself, um, the same procedure will apply if that, if the nodes were multilingual in Drupal 7, they will, like the [00:32:00] translations will also be ported over. But for the context of the example, we will not be covering multilingual migrations.

[00:32:21] So I think folks will learn a lot from this series that they could apply to, uh, things that aren't necessarily covered and it would be impossible to cover everything. I keep saying that I think everybody can learn something from this series. Um, so I'm curious, did you learn something from this series, you know, in, in doing this?

[00:33:04] But also because you want to translate that, you know, sometimes very technical, uh, knowledge into something that can be easily digestible, but, you know, by, by anyone. Um, and in general, that, um, there is, uh, a book called, uh, CSS Secrets. The author is Lea Verou and in the preface, she has a quote that I really like.

[00:33:59] Yes. [00:34:00] But ultimately my goal is to teach you how to do it, to teach you how to approach a problem that I might not be covering, to teach you how to debug, to give you an idea of, Oh, I need to use a process plugin for this, or maybe this is better suited with a source plugin or, um, Oh, yes, I didn't cover this specific entity type, but I get a sense of how I can go about understanding how it worked in Drupal 7 and I want, what I want to do in the data with Drupal 10.

[00:34:47] Michael Meyers: I think you do an awesome job of doing that. Every post I've read has said it's, it, you know, it's, it's really technical and in depth, but it's interesting and easy to read.

[00:35:15] To ensure that they're, you know, budgeting, scheduling, executing against this component. Mauricio, thank you so much for taking the time to put together this series. I know how busy you are on projects and how much effort it takes to write content of that quality. Uh, really appreciate you taking your time out of your day to join us today.

[00:36:08] Mauricio Dinarte: Thank you. See you.