On Magento 2.3.2, setup:upgrade command removes tables of disabled modules

[staradzinau]

Preconditions (*)

1. Magento 2.3.2 & 2.3-develop, 2.3.2-p1 (CE or EE- doesn't matter)
2. At least one installed module, which uses declarative schema

Steps to reproduce (*)

1. Disable module, which uses declarative schema, using console command (module:disable). For example, native Magento_ReleaseNotification extension
2. Run setup:upgrade command
3. Check in the database, tables of the disabled module from the first step (for Magento_ReleaseNotification - that's the only table 'release_notification_viewer_log')

Expected result (*)

1. Tables of the disabled module have been already removed

Actual result (*)

1. The database still contains tables of the disabled module, with all data.
2. See magento 2.3.1 command module:disable removes tables as well #24458 (comment)

[m2-assistant]

Hi @avgust1111. Thank you for your report.
To help us process this issue please make sure that you provided the following information:

• Summary of the issue
• Information on your environment
• Steps to reproduce
• Expected and actual results

Please make sure that the issue is reproducible on the vanilla Magento instance following Steps to reproduce. To deploy vanilla Magento instance on our environment, please, add a comment to the issue:

@magento give me 2.3-develop instance - upcoming 2.3.x release

For more details, please, review the Magento Contributor Assistant documentation.

@avgust1111 do you confirm that you were able to reproduce the issue on vanilla Magento instance following steps to reproduce?

• yes
• no

---

• Join Magento Community Engineering Slack and ask your questions in #github channel.

[staradzinau]

@magento give me 2.3-develop instance

[magento-engcom-team]

Hi @avgust1111. Thank you for your request. I'm working on Magento 2.3-develop instance for you

[magento-engcom-team]

Hi @avgust1111, here is your Magento instance.
Admin access: https://i-24926-2-3-develop.instances.magento-community.engineering/admin
Login: admin Password: 123123q
Instance will be terminated in up to 3 hours.

[staradzinau]

do you confirm that you were able to reproduce the issue on vanilla Magento instance following steps to reproduce?

Yes, I've reproduced the bug on my local environment. Unfortunately, I can't reproduce the issue on the test Magento instance, specified above, because I need SSH credentials to run console commands, specified above

It would be very nice to sort that issue out as soon as possible

[m2-assistant]

Hi @milindsingh. Thank you for working on this issue.
In order to make sure that issue has enough information and ready for development, please read and check the following instruction: 👇

• 1. Verify that issue has all the required information. (Preconditions, Steps to reproduce, Expected result, Actual result).

  Details

  If the issue has a valid description, the label Issue: Format is valid will be added to the issue automatically. Please, edit issue description if needed, until label Issue: Format is valid appears.

• 2. Verify that issue has a meaningful description and provides enough information to reproduce the issue. If the report is valid, add Issue: Clear Description label to the issue by yourself.

• 3. Add Component: XXXXX label(s) to the ticket, indicating the components it may be related to.

• 4. Verify that the issue is reproducible on 2.3-develop branch

  Details

  - Add the comment @magento give me 2.3-develop instance to deploy test instance on Magento infrastructure.
  - If the issue is reproducible on 2.3-develop branch, please, add the label Reproduced on 2.3.x.
  - If the issue is not reproducible, add your comment that issue is not reproducible and close the issue and stop verification process here!

---

• Join Magento Community Engineering Slack and ask your questions in #github channel.

[ghost]

After disabling module, please enable safe mode, while executing "setup:upgrade" or "setup:install", as this will creates a data dump during the installation or upgrade process. After this please restore the database while enabling the module.

Please follow the commands for same :

"setup:upgrade --safe-mode=1" : run after disabling the mod, this will create database backup.

"setup:upgrade --data-restore=1" : to restore database again after enabling the module.

Please refer Magento doc. https://devdocs.magento.com/guides/v2.3/extension-dev-guide/declarative-schema/migration-commands.html

[staradzinau]

Hello, @webkul-deepak
Is that a final solution, or just a temporary one, until the bug won't be fixed?
Anyway, is that solution suitable for the situation, when module is enabled again after some period of time? Will database restoring affect all changes in all tables (both data and schema), or only tables, related to the enabled module?

[m2-assistant]

Hi @webkul-deepak. Thank you for working on this issue.
In order to make sure that issue has enough information and ready for development, please read and check the following instruction: 👇

• 1. Verify that issue has all the required information. (Preconditions, Steps to reproduce, Expected result, Actual result).

  Details

  If the issue has a valid description, the label Issue: Format is valid will be added to the issue automatically. Please, edit issue description if needed, until label Issue: Format is valid appears.

• 2. Verify that issue has a meaningful description and provides enough information to reproduce the issue. If the report is valid, add Issue: Clear Description label to the issue by yourself.

• 3. Add Component: XXXXX label(s) to the ticket, indicating the components it may be related to.

• 4. Verify that the issue is reproducible on 2.3-develop branch

  Details

  - Add the comment @magento give me 2.3-develop instance to deploy test instance on Magento infrastructure.
  - If the issue is reproducible on 2.3-develop branch, please, add the label Reproduced on 2.3.x.
  - If the issue is not reproducible, add your comment that issue is not reproducible and close the issue and stop verification process here!

---

• Join Magento Community Engineering Slack and ask your questions in #github channel.

[m2-assistant]

Hi @engcom-Charlie. Thank you for working on this issue.
In order to make sure that issue has enough information and ready for development, please read and check the following instruction: 👇

• 1. Verify that issue has all the required information. (Preconditions, Steps to reproduce, Expected result, Actual result).

  Details

  If the issue has a valid description, the label Issue: Format is valid will be added to the issue automatically. Please, edit issue description if needed, until label Issue: Format is valid appears.

• 2. Verify that issue has a meaningful description and provides enough information to reproduce the issue. If the report is valid, add Issue: Clear Description label to the issue by yourself.

• 3. Add Component: XXXXX label(s) to the ticket, indicating the components it may be related to.

• 4. Verify that the issue is reproducible on 2.3-develop branch

  Details

  - Add the comment @magento give me 2.3-develop instance to deploy test instance on Magento infrastructure.
  - If the issue is reproducible on 2.3-develop branch, please, add the label Reproduced on 2.3.x.
  - If the issue is not reproducible, add your comment that issue is not reproducible and close the issue and stop verification process here!

• 5. Add label Issue: Confirmed once verification is complete.

• 6. Make sure that automatic system confirms that report has been added to the backlog.

[ghost]

Hi @avgust1111 the provided solution is not the temporary, it's the final solution. As it's the feature of Declarative Schema.
Yes, it will work same for modules which you've enabled after a long time.
Also "--data-restore=1" restores both data and schema, after enabling the module again.

[staradzinau]

Hello, @webkul-deepak.
Thank you for the update.
One moment is still unclear enough.

Will database restoring affect all changes in all tables (both data and schema), or only tables, related to the enabled module?

"--data-restore=1" restores both data and schema,

Will this action affect the whole database, or only tables, related to the module, which is enabled again?

[magento-engcom-team]

✅ Confirmed by @engcom-Charlie
Thank you for verifying the issue. Based on the provided information internal tickets MC-22908 were created

Issue Available: @engcom-Charlie, You will be automatically unassigned. Contributors/Maintainers can claim this issue to continue. To reclaim and continue work, reassign the ticket to yourself.

[twosg]

Personally I would prefer letting the "disabled" table(s) and data untouched while running setup:upgrade after disabling modules. If you really would want to delete the table and its (possibly important) data, you could run setup:upgrade --cleanup_mode=1 or something similar. Because then you know that you do not want the data.

[danemacmillan]

I get that this is the point behind Declarative Schema, but there should still be an option to prevent data from getting purged at all, like @twosg commented. This is a really nuclear default; it should require more deliberate input from users instead.

Removing and re-adding the data using the --safe-mode and --data-restore options just seem like a more error-prone, timely, and generally worse option than a simple --leave-my-shit-in-place option.

By the way, I found this discussion from the documentation.

[kanevbg]

Maybe better solution is that module:uninstall Module_Name is to remove the data, and not setup:upgrade?
EDIT: Eventually deferred DB data removal mechanism can be added, eg. when module is uninstalled, it can be 'queued' for DB data removal when running setup:upgrade. The queue in this case should be easy to manage/edit.

[gabrieldagama]

Disabling a module should not lead to the deletion of the database table or its contents, as described by some contributors above.

Closing this issue since the current behavior is the expected one.

Thanks.

[staradzinau]

@gabrieldagama Please, note: earlier, documentation has been updated: https://github.com/magento/devdocs/issues/5629. I suppose if the behaviour was changed, it's necessary to update the documentation, what do you think?

[gabrieldagama]

Hi @avgust1111, thanks for highlighting. Did some investigation on this and the current behavior is the expected behavior on this issue. I guess the documentation is still correct. I will check internally, but I still think that shouldn't be the behavior, maybe there is a reason for that.

Thanks again for your feedback.

[LiamKarlMitchell]

Can you not delete tables, its rather annoying.
Delete the table if the module is completely deleted or add a warning on every setup:upgrade that the tables still exist...

[barryvdh]

This is understandable from a technical point of view, but I would say this is wildly dangerous behaviour. I think you would usually almost never want to drop actual tables/columns on an upgrade, especially with automated deployments. Examples:

• A module is conflicting, so you disable modules one by one to find a solution -> your settings and data will now be dropped after deploying
• For some reason a module is missing temporarily to wait for compat with newer versions -> all old data is lost
• Technical error that disables the wrong module on deploy on a different branch, etc etc

I understand that you want to keep your database clean, but then please create an uninstall command. Not sure of the technical possibilities, but perhaps it's possible to keep the declarative schemas in the var folder, so if a module is deleted, the already used schemas are still available and can be used to actually revert a module if required (eg. by deleting that file also).

(I just noticed this because I disabled 2FA to let someone debug an issue on a staging enviroment, but after re-enabling: all the 2fa tokens were reset)

[bmelman]

This is indeed very dangerous and unexpected behaviour.
Disabling a module is not the same as removing a module: for removal (using i.e. composer) the removing of related tables would be logical.
That said at least an option to prevent this table removal behaviour (from module source code/config instead of cli) would be enough to even consider this in production.

So for now regressing back to Install/Upgradeschema

[nordcomputer]

Ok, just a question.
So, I disable a module and run setup:upgrade --safe-mode=1
After some time, I want to disable another module. So, I disable the second module and run setup:upgrade --safe-mode=1 again.

What happens, if I reactivate the first module and run setup:upgrade --data-restore=1? Will it restore the data from the first deactivated module or will it restore the data for both modules (even, if the second module is still deactivated)?

Bonus question: Once I have done setup:upgrade --safe-mode=1, would I have to do it everytime, when I run the command? In other words: will the backup get deleted, when I run the command later without the --safe-mode option?

[ghost]

@nordcomputer regarding : What happens, if I reactivate the first module and run setup:upgrade --data-restore=1? Will it restore the data from the first deactivated module or will it restore the data for both modules (even, if the second module is still deactivated)?
==> It only restore data for enabled module. It didn't restore data of deactivated modules.

regarding : Bonus question: Once I have done setup:upgrade --safe-mode=1, would I have to do it everytime, when I run the command? In other words: will the backup get deleted, when I run the command later without the --safe-mode option?
==> Whenever you need to run setup:upgrade after deactivating any module, in that case you need to execute it with --safe-mode=1 (to create database backup of deactivated module). In case if you're not deactivating any module, in that case their is no need to use --safe-mode=1. Also you can use --safe-mode=1 --data-restore=1 together, to create database backup and restore on the same time.