astlinux-devel Mailing List for AstLinux

Devs,

I enabled the e2fsck progress bar ...
https://github.com/astlinux-project/astlinux/commit/75d25893e712792192414931481631d1d8dc3c7c

For those of you building custom images, be sure to remove "initrd.img" so the initrd gets rebuilt.

I tested this on a net5501, Qotom Q190G4N-S07 (video console), Jetway NF9HG-2930, PC Engines APU2 and Lanner LEC-7220-N4 .  I pulled the power and saw e2fsck fix with the progress bar.

Additionally, as a side benefit, now whether the filesystem was just "cleaned" or also "modified" is shown.

Thanks for all the comments and feedback.

Lonnie

> Am 30.08.2017 um 05:58 schrieb Lonnie Abelbeck <li...@lo... <mailto:li...@lo...>>:
> 
> David,
> 
> Yes you can test .. if you "pull the power" on your box (or power off a VM without a proper shutdown) then use the RUNNIX "shell" to perform:
> --
> e2fsck -y -C 0 /dev/sda2
> --
> and if you have a separate /mnt/kd partition
> --
> e2fsck -y -C 0 /dev/sda3
> --
> 
> Using a VM is so fast the progress bar only flashes up for a second.
> 
> I also tested on an APU2, and it worked as expected.  A little more output when the filesystem is not clean, but should not be confusing to users.
> 
> Lonnie

Works fine for me in a VirtualBox VM. It took about 2-3 seconds.



> On Aug 29, 2017, at 9:31 PM, David Kerr <Da...@Ke... <mailto:Da...@Ke...>> wrote:
> 
>> I think using something that is formally part of the application is the best approach.  Is there anyway to test it?
>> 
>> David
>> 
>> On Tue, Aug 29, 2017 at 9:37 PM, Lonnie Abelbeck <li...@lo... <mailto:li...@lo...>> wrote:
>> Well, it turns out e2fsck has a "progress" bar feature, simply add "-C 0" to the command line.
>> 
>> I'm not sure how this will look when a serial console is attached mid-way through the progress, but there will be activity and it won't look like the box is hung.
>> 
>> So if we replace the current
>> --
>> e2fsck -y $ASTURW >/dev/null
>> -- with --
>> e2fsck -y -C 0 $ASTURW
>> --
>> we will get an extra line for the typical case, and several more lines and possibly a progress bar for the "dirty" case.
>> 
>> Is this the best approach ?
>> 
>> Lonnie
>> 
>> 
>> On Aug 29, 2017, at 5:56 PM, The Cadillac Kid via Astlinux-devel <ast...@li... <mailto:ast...@li...>> wrote:
>> 
>>> I thought fsck printed out as it goes?  with the -v option?  does the version in astlinux not?
>>> -Christopher
>>> 
>>> From: Lonnie Abelbeck <li...@lo... <mailto:li...@lo...>>
>>> To: AstLinux Developers Mailing List <ast...@li... <mailto:ast...@li...>>
>>> Sent: Tuesday, August 29, 2017 5:52 PM
>>> Subject: Re: [Astlinux-devel] Adding a "spinner" indicator when e2fsck is working
>>> 
>>> Hi Michael,
>>> 
>>> First, this issue that Tim Turpin reported today does not occur very often, but as larger SSD's become less expensive and easier to find, the time it takes to do a fsck after a hard power failure or corruption will become longer and longer.
>>> 
>>> Consider Tim's case, he notices his AstLinux box is not up, power is on ... so he checks the console ... unless there is some "activity" on the console it would appear the box is locked-up, but actually e2fsck is fixing his drive.
>>> 
>>> I don't think some static text before the e2fsck would help too much, easily missed and the asynchronous kernel logs make things more confusing coming in after the e2fsck log.
>>> 
>>> Lonnie
>>> 
>>> 
>>> On Aug 29, 2017, at 5:18 PM, Michael Knill <mic...@ip... <mailto:mic...@ip...>> wrote:
>>> 
>>>> Do you really need a spinner?
>>>> Couldn't you just echo a comment that e2fsk is working and this could take a REAL long time so don't panic?
>>>> You could then have another comment after e2fsk saying its done!
>>>> 
>>>> Regards
>>>> Michael Knill
>>>> 
>>>> -----Original Message-----
>>>> From: Lonnie Abelbeck <li...@lo... <mailto:li...@lo...>>
>>>> Reply-To: AstLinux Developers Mailing List <ast...@li... <mailto:ast...@li...>>
>>>> Date: Wednesday, 30 August 2017 at 7:58 am
>>>> To: AstLinux Developers Mailing List <ast...@li... <mailto:ast...@li...>>
>>>> Subject: Re: [Astlinux-devel] Adding a "spinner" indicator when e2fsck is working
>>>> 
>>>> 
>>>> On Aug 29, 2017, at 1:42 PM, Michael Keuter <li...@mk... <mailto:li...@mk...>> wrote:
>>>> 
>>>>>>> 
>>>>>>> Lonnie, don't give up so quickly :-).
>>>>>>> What you have already is quite a good starting point.
>>>>>>> What about adding "something needed" to our initrd busybox.config …
>>>>>>> 
>>>>>>> Michael
>>>>>> 
>>>>>> The initrd busybox config is not the problem, this is a fundamental problem using shell, though easily solved with bash, but bash is not a practical option with our initrd.
>>>>>> 
>>>>>> Get exit status of process that's piped to another
>>>>>> https://unix.stackexchange.com/questions/14270/get-exit-status-of-process-thats-piped-to-another <https://unix.stackexchange.com/questions/14270/get-exit-status-of-process-thats-piped-to-another>
>>>>>> 
>>>>>> I dont see an elegant solution.
>>>>>> 
>>>>>> My only other thought would be to create a background "spinner" process, then call "e2fsck -y $1 >/dev/null" and status="$?" then stop the background spinner process.
>>>>> 
>>>>> Sounds not so bad, at least worth a test.
>>>> 
>>>> Here is a working test of a "spinner" function:
>>>> https://gist.github.com/abelbeck/40c1da054ae6740f03e09ef70452cf70 <https://gist.github.com/abelbeck/40c1da054ae6740f03e09ef70452cf70>
>>>> 
>>>> In our case, instead of the test examples:
>>>> --
>>>> date
>>>> sleep 2
>>>> date
>>>> sleep 10
>>>> --
>>>> we would have:
>>>> --
>>>> e2fsck -y $ASTURW >/dev/null
>>>> status=$?
>>>> --
>>>> 
>>>> Seems to work, the question is whether it is worth adding.
>>>> 
>>>> Lonnie


Michael

http://www.mksolutions.info

David,

Yes you can test .. if you "pull the power" on your box (or power off a VM without a proper shutdown) then use the RUNNIX "shell" to perform:
--
e2fsck -y -C 0 /dev/sda2
--
and if you have a separate /mnt/kd partition
--
e2fsck -y -C 0 /dev/sda3
--

Using a VM is so fast the progress bar only flashes up for a second.

I also tested on an APU2, and it worked as expected.  A little more output when the filesystem is not clean, but should not be confusing to users.

Lonnie



On Aug 29, 2017, at 9:31 PM, David Kerr <Da...@Ke...> wrote:

> I think using something that is formally part of the application is the best approach.  Is there anyway to test it?
> 
> David
> 
> On Tue, Aug 29, 2017 at 9:37 PM, Lonnie Abelbeck <li...@lo...> wrote:
> Well, it turns out e2fsck has a "progress" bar feature, simply add "-C 0" to the command line.
> 
> I'm not sure how this will look when a serial console is attached mid-way through the progress, but there will be activity and it won't look like the box is hung.
> 
> So if we replace the current
> --
> e2fsck -y $ASTURW >/dev/null
> -- with --
> e2fsck -y -C 0 $ASTURW
> --
> we will get an extra line for the typical case, and several more lines and possibly a progress bar for the "dirty" case.
> 
> Is this the best approach ?
> 
> Lonnie
> 
> 
> On Aug 29, 2017, at 5:56 PM, The Cadillac Kid via Astlinux-devel <ast...@li...> wrote:
> 
> > I thought fsck printed out as it goes?  with the -v option?  does the version in astlinux not?
> > -Christopher
> >
> > From: Lonnie Abelbeck <li...@lo...>
> > To: AstLinux Developers Mailing List <ast...@li...>
> > Sent: Tuesday, August 29, 2017 5:52 PM
> > Subject: Re: [Astlinux-devel] Adding a "spinner" indicator when e2fsck is working
> >
> > Hi Michael,
> >
> > First, this issue that Tim Turpin reported today does not occur very often, but as larger SSD's become less expensive and easier to find, the time it takes to do a fsck after a hard power failure or corruption will become longer and longer.
> >
> > Consider Tim's case, he notices his AstLinux box is not up, power is on ... so he checks the console ... unless there is some "activity" on the console it would appear the box is locked-up, but actually e2fsck is fixing his drive.
> >
> > I don't think some static text before the e2fsck would help too much, easily missed and the asynchronous kernel logs make things more confusing coming in after the e2fsck log.
> >
> > Lonnie
> >
> >
> > On Aug 29, 2017, at 5:18 PM, Michael Knill <mic...@ip...> wrote:
> >
> > > Do you really need a spinner?
> > > Couldn't you just echo a comment that e2fsk is working and this could take a REAL long time so don't panic?
> > > You could then have another comment after e2fsk saying its done!
> > >
> > > Regards
> > > Michael Knill
> > >
> > > -----Original Message-----
> > > From: Lonnie Abelbeck <li...@lo...>
> > > Reply-To: AstLinux Developers Mailing List <ast...@li...>
> > > Date: Wednesday, 30 August 2017 at 7:58 am
> > > To: AstLinux Developers Mailing List <ast...@li...>
> > > Subject: Re: [Astlinux-devel] Adding a "spinner" indicator when e2fsck is working
> > >
> > >
> > > On Aug 29, 2017, at 1:42 PM, Michael Keuter <li...@mk...> wrote:
> > >
> > >>>>
> > >>>> Lonnie, don't give up so quickly :-).
> > >>>> What you have already is quite a good starting point.
> > >>>> What about adding "something needed" to our initrd busybox.config …
> > >>>>
> > >>>> Michael
> > >>>
> > >>> The initrd busybox config is not the problem, this is a fundamental problem using shell, though easily solved with bash, but bash is not a practical option with our initrd.
> > >>>
> > >>> Get exit status of process that's piped to another
> > >>> https://unix.stackexchange.com/questions/14270/get-exit-status-of-process-thats-piped-to-another
> > >>>
> > >>> I dont see an elegant solution.
> > >>>
> > >>> My only other thought would be to create a background "spinner" process, then call "e2fsck -y $1 >/dev/null" and status="$?" then stop the background spinner process.
> > >>
> > >> Sounds not so bad, at least worth a test.
> > >
> > > Here is a working test of a "spinner" function:
> > > https://gist.github.com/abelbeck/40c1da054ae6740f03e09ef70452cf70
> > >
> > > In our case, instead of the test examples:
> > > --
> > > date
> > > sleep 2
> > > date
> > > sleep 10
> > > --
> > > we would have:
> > > --
> > > e2fsck -y $ASTURW >/dev/null
> > > status=$?
> > > --
> > >
> > > Seems to work, the question is whether it is worth adding.
> > >
> > > Lonnie
> 
> 
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> Astlinux-devel mailing list
> Ast...@li...
> https://lists.sourceforge.net/lists/listinfo/astlinux-devel
> 
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot_______________________________________________
> Astlinux-devel mailing list
> Ast...@li...
> https://lists.sourceforge.net/lists/listinfo/astlinux-devel

I think using something that is formally part of the application is the
best approach.  Is there anyway to test it?

David

On Tue, Aug 29, 2017 at 9:37 PM, Lonnie Abelbeck <li...@lo...>
wrote:

> Well, it turns out e2fsck has a "progress" bar feature, simply add "-C 0"
> to the command line.
>
> I'm not sure how this will look when a serial console is attached mid-way
> through the progress, but there will be activity and it won't look like the
> box is hung.
>
> So if we replace the current
> --
> e2fsck -y $ASTURW >/dev/null
> -- with --
> e2fsck -y -C 0 $ASTURW
> --
> we will get an extra line for the typical case, and several more lines and
> possibly a progress bar for the "dirty" case.
>
> Is this the best approach ?
>
> Lonnie
>
>
> On Aug 29, 2017, at 5:56 PM, The Cadillac Kid via Astlinux-devel <
> ast...@li...> wrote:
>
> > I thought fsck printed out as it goes?  with the -v option?  does the
> version in astlinux not?
> > -Christopher
> >
> > From: Lonnie Abelbeck <li...@lo...>
> > To: AstLinux Developers Mailing List <astlinux-devel@lists.
> sourceforge.net>
> > Sent: Tuesday, August 29, 2017 5:52 PM
> > Subject: Re: [Astlinux-devel] Adding a "spinner" indicator when e2fsck
> is working
> >
> > Hi Michael,
> >
> > First, this issue that Tim Turpin reported today does not occur very
> often, but as larger SSD's become less expensive and easier to find, the
> time it takes to do a fsck after a hard power failure or corruption will
> become longer and longer.
> >
> > Consider Tim's case, he notices his AstLinux box is not up, power is on
> ... so he checks the console ... unless there is some "activity" on the
> console it would appear the box is locked-up, but actually e2fsck is fixing
> his drive.
> >
> > I don't think some static text before the e2fsck would help too much,
> easily missed and the asynchronous kernel logs make things more confusing
> coming in after the e2fsck log.
> >
> > Lonnie
> >
> >
> > On Aug 29, 2017, at 5:18 PM, Michael Knill <michael.knill@ipcsolutions.
> com.au> wrote:
> >
> > > Do you really need a spinner?
> > > Couldn't you just echo a comment that e2fsk is working and this could
> take a REAL long time so don't panic?
> > > You could then have another comment after e2fsk saying its done!
> > >
> > > Regards
> > > Michael Knill
> > >
> > > -----Original Message-----
> > > From: Lonnie Abelbeck <li...@lo...>
> > > Reply-To: AstLinux Developers Mailing List <astlinux-devel@lists.
> sourceforge.net>
> > > Date: Wednesday, 30 August 2017 at 7:58 am
> > > To: AstLinux Developers Mailing List <astlinux-devel@lists.
> sourceforge.net>
> > > Subject: Re: [Astlinux-devel] Adding a "spinner" indicator when e2fsck
> is working
> > >
> > >
> > > On Aug 29, 2017, at 1:42 PM, Michael Keuter <li...@mk...>
> wrote:
> > >
> > >>>>
> > >>>> Lonnie, don't give up so quickly :-).
> > >>>> What you have already is quite a good starting point.
> > >>>> What about adding "something needed" to our initrd busybox.config …
> > >>>>
> > >>>> Michael
> > >>>
> > >>> The initrd busybox config is not the problem, this is a fundamental
> problem using shell, though easily solved with bash, but bash is not a
> practical option with our initrd.
> > >>>
> > >>> Get exit status of process that's piped to another
> > >>> https://unix.stackexchange.com/questions/14270/get-exit-
> status-of-process-thats-piped-to-another
> > >>>
> > >>> I dont see an elegant solution.
> > >>>
> > >>> My only other thought would be to create a background "spinner"
> process, then call "e2fsck -y $1 >/dev/null" and status="$?" then stop the
> background spinner process.
> > >>
> > >> Sounds not so bad, at least worth a test.
> > >
> > > Here is a working test of a "spinner" function:
> > > https://gist.github.com/abelbeck/40c1da054ae6740f03e09ef70452cf70
> > >
> > > In our case, instead of the test examples:
> > > --
> > > date
> > > sleep 2
> > > date
> > > sleep 10
> > > --
> > > we would have:
> > > --
> > > e2fsck -y $ASTURW >/dev/null
> > > status=$?
> > > --
> > >
> > > Seems to work, the question is whether it is worth adding.
> > >
> > > Lonnie
>
>
> ------------------------------------------------------------
> ------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> Astlinux-devel mailing list
> Ast...@li...
> https://lists.sourceforge.net/lists/listinfo/astlinux-devel
>

Well, it turns out e2fsck has a "progress" bar feature, simply add "-C 0" to the command line.

I'm not sure how this will look when a serial console is attached mid-way through the progress, but there will be activity and it won't look like the box is hung.

So if we replace the current
--
e2fsck -y $ASTURW >/dev/null
-- with --
e2fsck -y -C 0 $ASTURW
--
we will get an extra line for the typical case, and several more lines and possibly a progress bar for the "dirty" case.

Is this the best approach ?

Lonnie


On Aug 29, 2017, at 5:56 PM, The Cadillac Kid via Astlinux-devel <ast...@li...> wrote:

> I thought fsck printed out as it goes?  with the -v option?  does the version in astlinux not?
> -Christopher
> 
> From: Lonnie Abelbeck <li...@lo...>
> To: AstLinux Developers Mailing List <ast...@li...> 
> Sent: Tuesday, August 29, 2017 5:52 PM
> Subject: Re: [Astlinux-devel] Adding a "spinner" indicator when e2fsck is working
> 
> Hi Michael,
> 
> First, this issue that Tim Turpin reported today does not occur very often, but as larger SSD's become less expensive and easier to find, the time it takes to do a fsck after a hard power failure or corruption will become longer and longer.
> 
> Consider Tim's case, he notices his AstLinux box is not up, power is on ... so he checks the console ... unless there is some "activity" on the console it would appear the box is locked-up, but actually e2fsck is fixing his drive.
> 
> I don't think some static text before the e2fsck would help too much, easily missed and the asynchronous kernel logs make things more confusing coming in after the e2fsck log.
> 
> Lonnie
> 
> 
> On Aug 29, 2017, at 5:18 PM, Michael Knill <mic...@ip...> wrote:
> 
> > Do you really need a spinner? 
> > Couldn't you just echo a comment that e2fsk is working and this could take a REAL long time so don't panic?
> > You could then have another comment after e2fsk saying its done!
> > 
> > Regards
> > Michael Knill
> > 
> > -----Original Message-----
> > From: Lonnie Abelbeck <li...@lo...>
> > Reply-To: AstLinux Developers Mailing List <ast...@li...>
> > Date: Wednesday, 30 August 2017 at 7:58 am
> > To: AstLinux Developers Mailing List <ast...@li...>
> > Subject: Re: [Astlinux-devel] Adding a "spinner" indicator when e2fsck is working
> > 
> > 
> > On Aug 29, 2017, at 1:42 PM, Michael Keuter <li...@mk...> wrote:
> > 
> >>>> 
> >>>> Lonnie, don't give up so quickly :-).
> >>>> What you have already is quite a good starting point.
> >>>> What about adding "something needed" to our initrd busybox.config …
> >>>> 
> >>>> Michael
> >>> 
> >>> The initrd busybox config is not the problem, this is a fundamental problem using shell, though easily solved with bash, but bash is not a practical option with our initrd.
> >>> 
> >>> Get exit status of process that's piped to another
> >>> https://unix.stackexchange.com/questions/14270/get-exit-status-of-process-thats-piped-to-another
> >>> 
> >>> I dont see an elegant solution.
> >>> 
> >>> My only other thought would be to create a background "spinner" process, then call "e2fsck -y $1 >/dev/null" and status="$?" then stop the background spinner process.
> >> 
> >> Sounds not so bad, at least worth a test.
> > 
> > Here is a working test of a "spinner" function:
> > https://gist.github.com/abelbeck/40c1da054ae6740f03e09ef70452cf70
> > 
> > In our case, instead of the test examples:
> > --
> > date
> > sleep 2
> > date
> > sleep 10
> > --
> > we would have:
> > --
> > e2fsck -y $ASTURW >/dev/null
> > status=$?
> > --
> > 
> > Seems to work, the question is whether it is worth adding.
> > 
> > Lonnie

Christopher,  AstLinux uses the latest e2fsprogs, we have all the options.

I don't have an example in front of me, but if it takes 15+ minutes for a "e2fsck -y /dev/sda2" then there must be a *lot* of fixes spewing on the screen, even without the -v option.  I'm not sure if opening up all of e2fsck's stdout is the best approach.

Lonnie



On Aug 29, 2017, at 5:56 PM, The Cadillac Kid via Astlinux-devel <ast...@li...> wrote:

> I thought fsck printed out as it goes?  with the -v option?  does the version in astlinux not?
> -Christopher
> 
> From: Lonnie Abelbeck <li...@lo...>
> To: AstLinux Developers Mailing List <ast...@li...> 
> Sent: Tuesday, August 29, 2017 5:52 PM
> Subject: Re: [Astlinux-devel] Adding a "spinner" indicator when e2fsck is working
> 
> Hi Michael,
> 
> First, this issue that Tim Turpin reported today does not occur very often, but as larger SSD's become less expensive and easier to find, the time it takes to do a fsck after a hard power failure or corruption will become longer and longer.
> 
> Consider Tim's case, he notices his AstLinux box is not up, power is on ... so he checks the console ... unless there is some "activity" on the console it would appear the box is locked-up, but actually e2fsck is fixing his drive.
> 
> I don't think some static text before the e2fsck would help too much, easily missed and the asynchronous kernel logs make things more confusing coming in after the e2fsck log.
> 
> Lonnie
> 
> 
> On Aug 29, 2017, at 5:18 PM, Michael Knill <mic...@ip...> wrote:
> 
> > Do you really need a spinner? 
> > Couldn't you just echo a comment that e2fsk is working and this could take a REAL long time so don't panic?
> > You could then have another comment after e2fsk saying its done!
> > 
> > Regards
> > Michael Knill
> > 
> > -----Original Message-----
> > From: Lonnie Abelbeck <li...@lo...>
> > Reply-To: AstLinux Developers Mailing List <ast...@li...>
> > Date: Wednesday, 30 August 2017 at 7:58 am
> > To: AstLinux Developers Mailing List <ast...@li...>
> > Subject: Re: [Astlinux-devel] Adding a "spinner" indicator when e2fsck is working
> > 
> > 
> > On Aug 29, 2017, at 1:42 PM, Michael Keuter <li...@mk...> wrote:
> > 
> >>>> 
> >>>> Lonnie, don't give up so quickly :-).
> >>>> What you have already is quite a good starting point.
> >>>> What about adding "something needed" to our initrd busybox.config …
> >>>> 
> >>>> Michael
> >>> 
> >>> The initrd busybox config is not the problem, this is a fundamental problem using shell, though easily solved with bash, but bash is not a practical option with our initrd.
> >>> 
> >>> Get exit status of process that's piped to another
> >>> https://unix.stackexchange.com/questions/14270/get-exit-status-of-process-thats-piped-to-another
> >>> 
> >>> I dont see an elegant solution.
> >>> 
> >>> My only other thought would be to create a background "spinner" process, then call "e2fsck -y $1 >/dev/null" and status="$?" then stop the background spinner process.
> >> 
> >> Sounds not so bad, at least worth a test.
> > 
> > Here is a working test of a "spinner" function:
> > https://gist.github.com/abelbeck/40c1da054ae6740f03e09ef70452cf70
> > 
> > In our case, instead of the test examples:
> > --
> > date
> > sleep 2
> > date
> > sleep 10
> > --
> > we would have:
> > --
> > e2fsck -y $ASTURW >/dev/null
> > status=$?
> > --
> > 
> > Seems to work, the question is whether it is worth adding.
> > 
> > Lonnie
> 
> 
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> Astlinux-devel mailing list
> Ast...@li...
> https://lists.sourceforge.net/lists/listinfo/astlinux-devel
> 
> 
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot_______________________________________________
> Astlinux-devel mailing list
> Ast...@li...
> https://lists.sourceforge.net/lists/listinfo/astlinux-devel

I thought fsck printed out as it goes?  with the -v option?  does the version in astlinux not?-Christopher

      From: Lonnie Abelbeck <li...@lo...>
 To: AstLinux Developers Mailing List <ast...@li...> 
 Sent: Tuesday, August 29, 2017 5:52 PM
 Subject: Re: [Astlinux-devel] Adding a "spinner" indicator when e2fsck is working
   
Hi Michael,

First, this issue that Tim Turpin reported today does not occur very often, but as larger SSD's become less expensive and easier to find, the time it takes to do a fsck after a hard power failure or corruption will become longer and longer.

Consider Tim's case, he notices his AstLinux box is not up, power is on ... so he checks the console ... unless there is some "activity" on the console it would appear the box is locked-up, but actually e2fsck is fixing his drive.

I don't think some static text before the e2fsck would help too much, easily missed and the asynchronous kernel logs make things more confusing coming in after the e2fsck log.

Lonnie


On Aug 29, 2017, at 5:18 PM, Michael Knill <mic...@ip...> wrote:

> Do you really need a spinner? 
> Couldn't you just echo a comment that e2fsk is working and this could take a REAL long time so don't panic?
> You could then have another comment after e2fsk saying its done!
> 
> Regards
> Michael Knill
> 
> -----Original Message-----
> From: Lonnie Abelbeck <li...@lo...>
> Reply-To: AstLinux Developers Mailing List <ast...@li...>
> Date: Wednesday, 30 August 2017 at 7:58 am
> To: AstLinux Developers Mailing List <ast...@li...>
> Subject: Re: [Astlinux-devel] Adding a "spinner" indicator when e2fsck is working
> 
> 
> On Aug 29, 2017, at 1:42 PM, Michael Keuter <li...@mk...> wrote:
> 
>>>> 
>>>> Lonnie, don't give up so quickly :-).
>>>> What you have already is quite a good starting point.
>>>> What about adding "something needed" to our initrd busybox.config …
>>>> 
>>>> Michael
>>> 
>>> The initrd busybox config is not the problem, this is a fundamental problem using shell, though easily solved with bash, but bash is not a practical option with our initrd.
>>> 
>>> Get exit status of process that's piped to another
>>> https://unix.stackexchange.com/questions/14270/get-exit-status-of-process-thats-piped-to-another
>>> 
>>> I dont see an elegant solution.
>>> 
>>> My only other thought would be to create a background "spinner" process, then call "e2fsck -y $1 >/dev/null" and status="$?" then stop the background spinner process.
>> 
>> Sounds not so bad, at least worth a test.
> 
> Here is a working test of a "spinner" function:
> https://gist.github.com/abelbeck/40c1da054ae6740f03e09ef70452cf70
> 
> In our case, instead of the test examples:
> --
> date
> sleep 2
> date
> sleep 10
> --
> we would have:
> --
> e2fsck -y $ASTURW >/dev/null
> status=$?
> --
> 
> Seems to work, the question is whether it is worth adding.
> 
> Lonnie


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Astlinux-devel mailing list
Ast...@li...
https://lists.sourceforge.net/lists/listinfo/astlinux-devel

   

Hi Michael,

First, this issue that Tim Turpin reported today does not occur very often, but as larger SSD's become less expensive and easier to find, the time it takes to do a fsck after a hard power failure or corruption will become longer and longer.

Consider Tim's case, he notices his AstLinux box is not up, power is on ... so he checks the console ... unless there is some "activity" on the console it would appear the box is locked-up, but actually e2fsck is fixing his drive.

I don't think some static text before the e2fsck would help too much, easily missed and the asynchronous kernel logs make things more confusing coming in after the e2fsck log.

Lonnie


On Aug 29, 2017, at 5:18 PM, Michael Knill <mic...@ip...> wrote:

> Do you really need a spinner? 
> Couldn't you just echo a comment that e2fsk is working and this could take a REAL long time so don't panic?
> You could then have another comment after e2fsk saying its done!
> 
> Regards
> Michael Knill
> 
> -----Original Message-----
> From: Lonnie Abelbeck <li...@lo...>
> Reply-To: AstLinux Developers Mailing List <ast...@li...>
> Date: Wednesday, 30 August 2017 at 7:58 am
> To: AstLinux Developers Mailing List <ast...@li...>
> Subject: Re: [Astlinux-devel] Adding a "spinner" indicator when e2fsck is working
> 
> 
> On Aug 29, 2017, at 1:42 PM, Michael Keuter <li...@mk...> wrote:
> 
>>>> 
>>>> Lonnie, don't give up so quickly :-).
>>>> What you have already is quite a good starting point.
>>>> What about adding "something needed" to our initrd busybox.config …
>>>> 
>>>> Michael
>>> 
>>> The initrd busybox config is not the problem, this is a fundamental problem using shell, though easily solved with bash, but bash is not a practical option with our initrd.
>>> 
>>> Get exit status of process that's piped to another
>>> https://unix.stackexchange.com/questions/14270/get-exit-status-of-process-thats-piped-to-another
>>> 
>>> I dont see an elegant solution.
>>> 
>>> My only other thought would be to create a background "spinner" process, then call "e2fsck -y $1 >/dev/null" and status="$?" then stop the background spinner process.
>> 
>> Sounds not so bad, at least worth a test.
> 
> Here is a working test of a "spinner" function:
> https://gist.github.com/abelbeck/40c1da054ae6740f03e09ef70452cf70
> 
> In our case, instead of the test examples:
> --
> date
> sleep 2
> date
> sleep 10
> --
> we would have:
> --
> e2fsck -y $ASTURW >/dev/null
> status=$?
> --
> 
> Seems to work, the question is whether it is worth adding.
> 
> Lonnie

Do you really need a spinner? 
Couldn't you just echo a comment that e2fsk is working and this could take a REAL long time so don't panic?
You could then have another comment after e2fsk saying its done!

Regards
Michael Knill

-----Original Message-----
From: Lonnie Abelbeck <li...@lo...>
Reply-To: AstLinux Developers Mailing List <ast...@li...>
Date: Wednesday, 30 August 2017 at 7:58 am
To: AstLinux Developers Mailing List <ast...@li...>
Subject: Re: [Astlinux-devel] Adding a "spinner" indicator when e2fsck is working


On Aug 29, 2017, at 1:42 PM, Michael Keuter <li...@mk...> wrote:

>>> 
>>> Lonnie, don't give up so quickly :-).
>>> What you have already is quite a good starting point.
>>> What about adding "something needed" to our initrd busybox.config …
>>> 
>>> Michael
>> 
>> The initrd busybox config is not the problem, this is a fundamental problem using shell, though easily solved with bash, but bash is not a practical option with our initrd.
>> 
>> Get exit status of process that's piped to another
>> https://unix.stackexchange.com/questions/14270/get-exit-status-of-process-thats-piped-to-another
>> 
>> I dont see an elegant solution.
>> 
>> My only other thought would be to create a background "spinner" process, then call "e2fsck -y $1 >/dev/null" and status="$?" then stop the background spinner process.
> 
> Sounds not so bad, at least worth a test.

Here is a working test of a "spinner" function:
https://gist.github.com/abelbeck/40c1da054ae6740f03e09ef70452cf70

In our case, instead of the test examples:
--
date
sleep 2
date
sleep 10
--
we would have:
--
e2fsck -y $ASTURW >/dev/null
status=$?
--

Seems to work, the question is whether it is worth adding.

Lonnie


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Astlinux-devel mailing list
Ast...@li...
https://lists.sourceforge.net/lists/listinfo/astlinux-devel

On Aug 29, 2017, at 1:42 PM, Michael Keuter <li...@mk...> wrote:

>>> 
>>> Lonnie, don't give up so quickly :-).
>>> What you have already is quite a good starting point.
>>> What about adding "something needed" to our initrd busybox.config …
>>> 
>>> Michael
>> 
>> The initrd busybox config is not the problem, this is a fundamental problem using shell, though easily solved with bash, but bash is not a practical option with our initrd.
>> 
>> Get exit status of process that's piped to another
>> https://unix.stackexchange.com/questions/14270/get-exit-status-of-process-thats-piped-to-another
>> 
>> I dont see an elegant solution.
>> 
>> My only other thought would be to create a background "spinner" process, then call "e2fsck -y $1 >/dev/null" and status="$?" then stop the background spinner process.
> 
> Sounds not so bad, at least worth a test.

Here is a working test of a "spinner" function:
https://gist.github.com/abelbeck/40c1da054ae6740f03e09ef70452cf70

In our case, instead of the test examples:
--
date
sleep 2
date
sleep 10
--
we would have:
--
e2fsck -y $ASTURW >/dev/null
status=$?
--

Seems to work, the question is whether it is worth adding.

Lonnie

On Aug 29, 2017, at 1:42 PM, Michael Keuter <li...@mk...> wrote:

> 
>> 
>> The initrd busybox config is not the problem, this is a fundamental problem using shell, though easily solved with bash, but bash is not a practical option with our initrd.
>> 
>> Get exit status of process that's piped to another
>> https://unix.stackexchange.com/questions/14270/get-exit-status-of-process-thats-piped-to-another
>> 
>> I dont see an elegant solution.
>> 
>> My only other thought would be to create a background "spinner" process, then call "e2fsck -y $1 >/dev/null" and status="$?" then stop the background spinner process.
> 
> Sounds not so bad, at least worth a test.
> BTW: Wasn't there a kind of mini-shell in busybox ("ash" or so)? 

Yes we are using the Busybox "ash" shell in the initrd.

Lonnie

Hi Christopher,

I guess that is an option as well, but as Tim reported 15 minutes+ of e2fsck spewing messages is not very useful either.

I seem to recall the mainline distros use some sort of spinner wrapper and hide the fsck messages while repairing.

Lonnie


On Aug 29, 2017, at 2:23 PM, The Cadillac Kid via Astlinux-devel <ast...@li...> wrote:

> why not just let the fsck details be printed to the primary console? 
> 
> 
> 
> From: Lonnie Abelbeck <li...@lo...>
> To: AstLinux Developers Mailing List <ast...@li...> 
> Sent: Tuesday, August 29, 2017 2:12 PM
> Subject: [Astlinux-devel] Adding a "spinner" indicator when e2fsck is working
> 
> Hi Devs,
> 
> David made a good point, so I looked into piping the output of e2fsck to a "spinner" shell function, triggered by a newline and update no more often then ever second ...
> --
> spinner()
> {
>   local i sp secs nsecs line IFS
> 
>   i=0
>   secs=0
> 
>   unset IFS
>   while read line; do
>     nsecs=$(cut -d. -f1 /proc/uptime)
>     if [ $nsecs -ne $secs ]; then
>       secs=$nsecs
>       i=$(((i+1)%4))
>       case $i in
>         1) sp='-' ;;
>         2) sp='\' ;;
>         3) sp='|' ;;
>         *) sp='/' ;;
>       esac
>       echo -ne " Working[${sp}] \r"
>     fi
>   done
> 
>   echo -ne "            \r"
> }
> --
> It took some work getting this to work without printf, no date, no bash ${str:1:1}, etc., but it works with the current Busybox configs.
> 
> But ... here are the two places "spinner" would apply to ...
> 
> 1) in the initrd, linuxrc ...
> --
>     e2fsck -y $ASTURW >/dev/null
>     status=$?
> 
>     case $status in
> --
> 
> 2) in astlinux, /etc/rc ...
> --
>   echo "Checking $1"
>   e2fsck -y $1 >/dev/null
> 
>   status="$?"
> 
>   case $status in
> --
> 
> The plan would be to use:
> --
> e2fsck -y $1 | spinner
> --
> But the return value of $? is no longer from e2fsck, but rather spinner
> 
> You might try ...
> --
> ( e2fsck -y $1 ; status="$?" ) | spinner
> --
> Almost, but now "status" is defined in a subshell which is not propagated up and lost.
> 
> A simple but ugly solution is to use a temp file ...
> --
> ( e2fsck -y $1 ; echo "$?" > /tmp/e2fsck_status ) | spinner
> status="$(cat /tmp/e2fsck_status)"
> rm /tmp/e2fsck_status
> --
> but not to mention the initrd does not currently mount a /tmp filesystem. We would have to have to create and delete a tmpfs filesystem just to get the return value from e2fsck. :-(
> 
> I'm leaning toward leaving things as is, comments ?
> 
> Lonnie
> 
> 
> On Aug 29, 2017, at 8:20 AM, Lonnie Abelbeck <li...@lo...> wrote:
> 
>> Hi David,
>> 
>> Agreed, no e2fsck output when it is forced to work for a long period is not ideal.  Thankfully this is a very rare occurrence.
>> 
>> If we can come up with a clever spinning wheel or such counting the lines e2fsck generates might be a good addition.
>> 
>> Recall this all occurs in the "initrd", so the available packages and BusyBox config is quite limited.
>> 
>> Any further discussion on this idea should move to the astlinux-devel list.
>> 
>> Lonnie
>> 
>> 
>> On Aug 29, 2017, at 7:48 AM, David Kerr <Da...@Ke...> wrote:
>> 
>>> Is that a good idea (redirecting e2fsck to null)?  15 minutes is a long time to wait with no indication of anything happening.
>>> 
>>> David
>>> 
>>> On Tue, Aug 29, 2017 at 8:33 AM, Lonnie Abelbeck <li...@lo...> wrote:
>>> Hi Tim,
>>> 
>>> Yes, it was running "e2fsck" with stdout redirected to /dev/null so you did not see it working ... eventually e2fsck will return with a result code if left long enough.
>>> 
>>> Good to hear your filesystem is now clean, you might reboot once again to make sure the filesystem is good.
>>> 
>>> BTW, if the automatic e2fsck repair did not work, here is the manual fallback procedure:
>>> --
>>> ## reboot, and quickly when the RUNNIX boot menu appears type "shell"
>>> boot: shell
>>> 
>>> ## Wait for a "runnix# " CLI prompt.
>>> ## determine the first Linux ext2 partition, usually always /dev/sda2
>>> runnix# findfs LABEL=ASTURW
>>> 
>>> ## using the findfs result, run e2fsck manually, you may want to add -y or -p options
>>> runnix# e2fsck /dev/sda2
>>> 
>>> ## output a list of options for e2fsck
>>> runnix# e2fsck
>>> Usage: e2fsck [-panyrcdfktvDFV] [-b superblock] [-B blocksize]
>>>                [-l|-L bad_blocks_file] [-C fd] [-j external_journal]
>>>                [-E extended-options] [-z undo_file] device
>>> 
>>> Emergency help:
>>> -p                   Automatic repair (no questions)
>>> -n                   Make no changes to the filesystem
>>> -y                   Assume "yes" to all questions
>>> -c                   Check for bad blocks and add them to the badblock list
>>> -f                   Force checking even if filesystem is marked clean
>>> -v                   Be verbose
>>> -b superblock        Use alternative superblock
>>> -B blocksize         Force blocksize when looking for superblock
>>> -j external_journal  Set location of the external journal
>>> -l bad_blocks_file   Add to badblocks list
>>> -L bad_blocks_file   Set badblocks list
>>> -z undo_file         Create an undo file
>>> 
>>> ## Depending on how you initially configured AstLinux, check if you also have a ASTKD partition
>>> ## (typically /dev/sda3 if it exists)
>>> runnix# findfs LABEL=ASTKD
>>> ## If the ASTKD label exists repeat the e2fsck steps above using the ASTKD label partition.
>>> 
>>> ## reboot by issuing "exit"
>>> runnix# exit
>>> --
>>> 
>>> Lonnie
>>> 
>>> 
>>> On Aug 29, 2017, at 7:02 AM, Tim Turpin <tt...@z-...> wrote:
>>> 
>>>> Never mind.  After sitting at that point for about 15 minutes, it somehow recovered and finished booting up.  All seems well now.
>>>> 
>>>> From: Tim Turpin [mailto:tt...@z-...]
>>>> Sent: Tuesday, August 29, 2017 7:34 AM
>>>> To: ast...@li...
>>>> Subject: [Astlinux-users] ASTLinux stopped booting
>>>> 
>>>> We took a power hit on our test system yesterday, and now it will only load up to a certain point and stops with the following screen:
> 
> <image001.jpeg>
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> Astlinux-devel mailing list
> Ast...@li...
> https://lists.sourceforge.net/lists/listinfo/astlinux-devel
> 
> 
> 
> <image001.jpeg>------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot_______________________________________________
> Astlinux-devel mailing list
> Ast...@li...
> https://lists.sourceforge.net/lists/listinfo/astlinux-devel

why not just let the fsck details be printed to the primary console? 



      From: Lonnie Abelbeck <li...@lo...>
 To: AstLinux Developers Mailing List <ast...@li...> 
 Sent: Tuesday, August 29, 2017 2:12 PM
 Subject: [Astlinux-devel] Adding a "spinner" indicator when e2fsck is working
   
Hi Devs,
David made a good point, so I looked into piping the output of e2fsck to a "spinner" shell function, triggered by a newline and update no more often then ever second ...--spinner()
{
  local i sp secs nsecs line IFS

  i=0
  secs=0

  unset IFS
  while read line; do
    nsecs=$(cut -d. -f1 /proc/uptime)
    if [ $nsecs -ne $secs ]; then
      secs=$nsecs
      i=$(((i+1)%4))
      case $i in
        1) sp='-' ;;
        2) sp='\' ;;
        3) sp='|' ;;
        *) sp='/' ;;
      esac
      echo -ne " Working[${sp}] \r"
    fi
  done

  echo -ne "            \r"
}
--It took some work getting this to work without printf, no date, no bash ${str:1:1}, etc., but it works with the current Busybox configs.
But ... here are the two places "spinner" would apply to ...
1) in the initrd, linuxrc ...--    e2fsck -y $ASTURW >/dev/null
    status=$?

    case $status in--
2) in astlinux, /etc/rc ...--  echo "Checking $1"
  e2fsck -y $1 >/dev/null

  status="$?"

  case $status in--
The plan would be to use:--e2fsck -y $1 | spinner--But the return value of $? is no longer from e2fsck, but rather spinner
You might try ...--( e2fsck -y $1 ; status="$?" ) | spinner--Almost, but now "status" is defined in a subshell which is not propagated up and lost.
A simple but ugly solution is to use a temp file ...--( e2fsck -y $1 ; echo "$?" > /tmp/e2fsck_status ) | spinnerstatus="$(cat /tmp/e2fsck_status)"rm /tmp/e2fsck_status--but not to mention the initrd does not currently mount a /tmp filesystem. We would have to have to create and delete a tmpfs filesystem just to get the return value from e2fsck. :-(
I'm leaning toward leaving things as is, comments ?
Lonnie

On Aug 29, 2017, at 8:20 AM, Lonnie Abelbeck <li...@lo...> wrote:


Hi David,

Agreed, no e2fsck output when it is forced to work for a long period is not ideal.  Thankfully this is a very rare occurrence.

If we can come up with a clever spinning wheel or such counting the lines e2fsck generates might be a good addition.

Recall this all occurs in the "initrd", so the available packages and BusyBox config is quite limited.

Any further discussion on this idea should move to the astlinux-devel list.

Lonnie


On Aug 29, 2017, at 7:48 AM, David Kerr <Da...@Ke...> wrote:


Is that a good idea (redirecting e2fsck to null)?  15 minutes is a long time to wait with no indication of anything happening.

David

On Tue, Aug 29, 2017 at 8:33 AM, Lonnie Abelbeck <li...@lo...> wrote:
Hi Tim,

Yes, it was running "e2fsck" with stdout redirected to /dev/null so you did not see it working ... eventually e2fsck will return with a result code if left long enough.

Good to hear your filesystem is now clean, you might reboot once again to make sure the filesystem is good.

BTW, if the automatic e2fsck repair did not work, here is the manual fallback procedure:
--
## reboot, and quickly when the RUNNIX boot menu appears type "shell"
boot: shell

## Wait for a "runnix# " CLI prompt.
## determine the first Linux ext2 partition, usually always /dev/sda2
runnix# findfs LABEL=ASTURW

## using the findfs result, run e2fsck manually, you may want to add -y or -p options
runnix# e2fsck /dev/sda2

## output a list of options for e2fsck
runnix# e2fsck
Usage: e2fsck [-panyrcdfktvDFV] [-b superblock] [-B blocksize]
               [-l|-L bad_blocks_file] [-C fd] [-j external_journal]
               [-E extended-options] [-z undo_file] device

Emergency help:
-p                   Automatic repair (no questions)
-n                   Make no changes to the filesystem
-y                   Assume "yes" to all questions
-c                   Check for bad blocks and add them to the badblock list
-f                   Force checking even if filesystem is marked clean
-v                   Be verbose
-b superblock        Use alternative superblock
-B blocksize         Force blocksize when looking for superblock
-j external_journal  Set location of the external journal
-l bad_blocks_file   Add to badblocks list
-L bad_blocks_file   Set badblocks list
-z undo_file         Create an undo file

## Depending on how you initially configured AstLinux, check if you also have a ASTKD partition
## (typically /dev/sda3 if it exists)
runnix# findfs LABEL=ASTKD
## If the ASTKD label exists repeat the e2fsck steps above using the ASTKD label partition.

## reboot by issuing "exit"
runnix# exit
--

Lonnie


On Aug 29, 2017, at 7:02 AM, Tim Turpin <tt...@z-...> wrote:


Never mind.  After sitting at that point for about 15 minutes, it somehow recovered and finished booting up.  All seems well now.

From: Tim Turpin [mailto:tt...@z-...]
Sent: Tuesday, August 29, 2017 7:34 AM
To: ast...@li...
Subject: [Astlinux-users] ASTLinux stopped booting

We took a power hit on our test system yesterday, and now it will only load up to a certain point and stops with the following screen:




------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot_______________________________________________
Astlinux-devel mailing list
Ast...@li...
https://lists.sourceforge.net/lists/listinfo/astlinux-devel


   

> Am 29.08.2017 um 20:30 schrieb Lonnie Abelbeck <li...@lo...>:
> 
> 
> On Aug 29, 2017, at 1:18 PM, Michael Keuter <li...@mk...> wrote:
> 
>> 
>>> Am 29.08.2017 um 20:12 schrieb Lonnie Abelbeck <li...@lo...>:
>>> 
>>> Hi Devs,
>>> 
>>> David made a good point, so I looked into piping the output of e2fsck to a "spinner" shell function, triggered by a newline and update no more often then ever second ...
>>> --
>>> spinner()
>>> {
>>> local i sp secs nsecs line IFS
>>> 
>>> i=0
>>> secs=0
>>> 
>>> unset IFS
>>> while read line; do
>>>   nsecs=$(cut -d. -f1 /proc/uptime)
>>>   if [ $nsecs -ne $secs ]; then
>>>     secs=$nsecs
>>>     i=$(((i+1)%4))
>>>     case $i in
>>>       1) sp='-' ;;
>>>       2) sp='\' ;;
>>>       3) sp='|' ;;
>>>       *) sp='/' ;;
>>>     esac
>>>     echo -ne " Working[${sp}] \r"
>>>   fi
>>> done
>>> 
>>> echo -ne "            \r"
>>> }
>>> --
>>> It took some work getting this to work without printf, no date, no bash ${str:1:1}, etc., but it works with the current Busybox configs.
>>> 
>>> But ... here are the two places "spinner" would apply to ...
>>> 
>>> 1) in the initrd, linuxrc ...
>>> --
>>>   e2fsck -y $ASTURW >/dev/null
>>>   status=$?
>>> 
>>>   case $status in
>>> --
>>> 
>>> 2) in astlinux, /etc/rc ...
>>> --
>>> echo "Checking $1"
>>> e2fsck -y $1 >/dev/null
>>> 
>>> status="$?"
>>> 
>>> case $status in
>>> --
>>> 
>>> The plan would be to use:
>>> --
>>> e2fsck -y $1 | spinner
>>> --
>>> But the return value of $? is no longer from e2fsck, but rather spinner
>>> 
>>> You might try ...
>>> --
>>> ( e2fsck -y $1 ; status="$?" ) | spinner
>>> --
>>> Almost, but now "status" is defined in a subshell which is not propagated up and lost.
>>> 
>>> A simple but ugly solution is to use a temp file ...
>>> --
>>> ( e2fsck -y $1 ; echo "$?" > /tmp/e2fsck_status ) | spinner
>>> status="$(cat /tmp/e2fsck_status)"
>>> rm /tmp/e2fsck_status
>>> --
>>> but not to mention the initrd does not currently mount a /tmp filesystem. We would have to have to create and delete a tmpfs filesystem just to get the return value from e2fsck. :-(
>>> 
>>> I'm leaning toward leaving things as is, comments ?
>>> 
>>> Lonnie
>>> 
>>> 
>>> On Aug 29, 2017, at 8:20 AM, Lonnie Abelbeck <li...@lo...> wrote:
>>> 
>>>> Hi David,
>>>> 
>>>> Agreed, no e2fsck output when it is forced to work for a long period is not ideal.  Thankfully this is a very rare occurrence.
>>>> 
>>>> If we can come up with a clever spinning wheel or such counting the lines e2fsck generates might be a good addition.
>>>> 
>>>> Recall this all occurs in the "initrd", so the available packages and BusyBox config is quite limited.
>>>> 
>>>> Any further discussion on this idea should move to the astlinux-devel list.
>>>> 
>>>> Lonnie
>>>> 
>>>> 
>>>> On Aug 29, 2017, at 7:48 AM, David Kerr <Da...@Ke...> wrote:
>>>> 
>>>>> Is that a good idea (redirecting e2fsck to null)?  15 minutes is a long time to wait with no indication of anything happening.
>>>>> 
>>>>> David
>>>>> 
>>>>> On Tue, Aug 29, 2017 at 8:33 AM, Lonnie Abelbeck <li...@lo...> wrote:
>>>>> Hi Tim,
>>>>> 
>>>>> Yes, it was running "e2fsck" with stdout redirected to /dev/null so you did not see it working ... eventually e2fsck will return with a result code if left long enough.
>>>>> 
>>>>> Good to hear your filesystem is now clean, you might reboot once again to make sure the filesystem is good.
>>>>> 
>>>>> BTW, if the automatic e2fsck repair did not work, here is the manual fallback procedure:
>>>>> --
>>>>> ## reboot, and quickly when the RUNNIX boot menu appears type "shell"
>>>>> boot: shell
>>>>> 
>>>>> ## Wait for a "runnix# " CLI prompt.
>>>>> ## determine the first Linux ext2 partition, usually always /dev/sda2
>>>>> runnix# findfs LABEL=ASTURW
>>>>> 
>>>>> ## using the findfs result, run e2fsck manually, you may want to add -y or -p options
>>>>> runnix# e2fsck /dev/sda2
>>>>> 
>>>>> ## output a list of options for e2fsck
>>>>> runnix# e2fsck
>>>>> Usage: e2fsck [-panyrcdfktvDFV] [-b superblock] [-B blocksize]
>>>>>              [-l|-L bad_blocks_file] [-C fd] [-j external_journal]
>>>>>              [-E extended-options] [-z undo_file] device
>>>>> 
>>>>> Emergency help:
>>>>> -p                   Automatic repair (no questions)
>>>>> -n                   Make no changes to the filesystem
>>>>> -y                   Assume "yes" to all questions
>>>>> -c                   Check for bad blocks and add them to the badblock list
>>>>> -f                   Force checking even if filesystem is marked clean
>>>>> -v                   Be verbose
>>>>> -b superblock        Use alternative superblock
>>>>> -B blocksize         Force blocksize when looking for superblock
>>>>> -j external_journal  Set location of the external journal
>>>>> -l bad_blocks_file   Add to badblocks list
>>>>> -L bad_blocks_file   Set badblocks list
>>>>> -z undo_file         Create an undo file
>>>>> 
>>>>> ## Depending on how you initially configured AstLinux, check if you also have a ASTKD partition
>>>>> ## (typically /dev/sda3 if it exists)
>>>>> runnix# findfs LABEL=ASTKD
>>>>> ## If the ASTKD label exists repeat the e2fsck steps above using the ASTKD label partition.
>>>>> 
>>>>> ## reboot by issuing "exit"
>>>>> runnix# exit
>>>>> --
>>>>> 
>>>>> Lonnie
>>>>> 
>>>>> 
>>>>> On Aug 29, 2017, at 7:02 AM, Tim Turpin <tt...@z-...> wrote:
>>>>> 
>>>>>> Never mind.  After sitting at that point for about 15 minutes, it somehow recovered and finished booting up.  All seems well now.
>>>>>> 
>>>>>> From: Tim Turpin [mailto:tt...@z-...]
>>>>>> Sent: Tuesday, August 29, 2017 7:34 AM
>>>>>> To: ast...@li...
>>>>>> Subject: [Astlinux-users] ASTLinux stopped booting
>>>>>> 
>>>>>> We took a power hit on our test system yesterday, and now it will only load up to a certain point and stops with the following screen:
>> 
>> Lonnie, don't give up so quickly :-).
>> What you have already is quite a good starting point.
>> What about adding "something needed" to our initrd busybox.config …
>> 
>> Michael
> 
> The initrd busybox config is not the problem, this is a fundamental problem using shell, though easily solved with bash, but bash is not a practical option with our initrd.
> 
> Get exit status of process that's piped to another
> https://unix.stackexchange.com/questions/14270/get-exit-status-of-process-thats-piped-to-another
> 
> I dont see an elegant solution.
> 
> My only other thought would be to create a background "spinner" process, then call "e2fsck -y $1 >/dev/null" and status="$?" then stop the background spinner process.

Sounds not so bad, at least worth a test.
BTW: Wasn't there a kind of mini-shell in busybox ("ash" or so)? 

> Lonnie

Michael

http://www.mksolutions.info

On Aug 29, 2017, at 1:18 PM, Michael Keuter <li...@mk...> wrote:

> 
>> Am 29.08.2017 um 20:12 schrieb Lonnie Abelbeck <li...@lo...>:
>> 
>> Hi Devs,
>> 
>> David made a good point, so I looked into piping the output of e2fsck to a "spinner" shell function, triggered by a newline and update no more often then ever second ...
>> --
>> spinner()
>> {
>>  local i sp secs nsecs line IFS
>> 
>>  i=0
>>  secs=0
>> 
>>  unset IFS
>>  while read line; do
>>    nsecs=$(cut -d. -f1 /proc/uptime)
>>    if [ $nsecs -ne $secs ]; then
>>      secs=$nsecs
>>      i=$(((i+1)%4))
>>      case $i in
>>        1) sp='-' ;;
>>        2) sp='\' ;;
>>        3) sp='|' ;;
>>        *) sp='/' ;;
>>      esac
>>      echo -ne " Working[${sp}] \r"
>>    fi
>>  done
>> 
>>  echo -ne "            \r"
>> }
>> --
>> It took some work getting this to work without printf, no date, no bash ${str:1:1}, etc., but it works with the current Busybox configs.
>> 
>> But ... here are the two places "spinner" would apply to ...
>> 
>> 1) in the initrd, linuxrc ...
>> --
>>    e2fsck -y $ASTURW >/dev/null
>>    status=$?
>> 
>>    case $status in
>> --
>> 
>> 2) in astlinux, /etc/rc ...
>> --
>>  echo "Checking $1"
>>  e2fsck -y $1 >/dev/null
>> 
>>  status="$?"
>> 
>>  case $status in
>> --
>> 
>> The plan would be to use:
>> --
>> e2fsck -y $1 | spinner
>> --
>> But the return value of $? is no longer from e2fsck, but rather spinner
>> 
>> You might try ...
>> --
>> ( e2fsck -y $1 ; status="$?" ) | spinner
>> --
>> Almost, but now "status" is defined in a subshell which is not propagated up and lost.
>> 
>> A simple but ugly solution is to use a temp file ...
>> --
>> ( e2fsck -y $1 ; echo "$?" > /tmp/e2fsck_status ) | spinner
>> status="$(cat /tmp/e2fsck_status)"
>> rm /tmp/e2fsck_status
>> --
>> but not to mention the initrd does not currently mount a /tmp filesystem. We would have to have to create and delete a tmpfs filesystem just to get the return value from e2fsck. :-(
>> 
>> I'm leaning toward leaving things as is, comments ?
>> 
>> Lonnie
>> 
>> 
>> On Aug 29, 2017, at 8:20 AM, Lonnie Abelbeck <li...@lo...> wrote:
>> 
>>> Hi David,
>>> 
>>> Agreed, no e2fsck output when it is forced to work for a long period is not ideal.  Thankfully this is a very rare occurrence.
>>> 
>>> If we can come up with a clever spinning wheel or such counting the lines e2fsck generates might be a good addition.
>>> 
>>> Recall this all occurs in the "initrd", so the available packages and BusyBox config is quite limited.
>>> 
>>> Any further discussion on this idea should move to the astlinux-devel list.
>>> 
>>> Lonnie
>>> 
>>> 
>>> On Aug 29, 2017, at 7:48 AM, David Kerr <Da...@Ke...> wrote:
>>> 
>>>> Is that a good idea (redirecting e2fsck to null)?  15 minutes is a long time to wait with no indication of anything happening.
>>>> 
>>>> David
>>>> 
>>>> On Tue, Aug 29, 2017 at 8:33 AM, Lonnie Abelbeck <li...@lo...> wrote:
>>>> Hi Tim,
>>>> 
>>>> Yes, it was running "e2fsck" with stdout redirected to /dev/null so you did not see it working ... eventually e2fsck will return with a result code if left long enough.
>>>> 
>>>> Good to hear your filesystem is now clean, you might reboot once again to make sure the filesystem is good.
>>>> 
>>>> BTW, if the automatic e2fsck repair did not work, here is the manual fallback procedure:
>>>> --
>>>> ## reboot, and quickly when the RUNNIX boot menu appears type "shell"
>>>> boot: shell
>>>> 
>>>> ## Wait for a "runnix# " CLI prompt.
>>>> ## determine the first Linux ext2 partition, usually always /dev/sda2
>>>> runnix# findfs LABEL=ASTURW
>>>> 
>>>> ## using the findfs result, run e2fsck manually, you may want to add -y or -p options
>>>> runnix# e2fsck /dev/sda2
>>>> 
>>>> ## output a list of options for e2fsck
>>>> runnix# e2fsck
>>>> Usage: e2fsck [-panyrcdfktvDFV] [-b superblock] [-B blocksize]
>>>>               [-l|-L bad_blocks_file] [-C fd] [-j external_journal]
>>>>               [-E extended-options] [-z undo_file] device
>>>> 
>>>> Emergency help:
>>>> -p                   Automatic repair (no questions)
>>>> -n                   Make no changes to the filesystem
>>>> -y                   Assume "yes" to all questions
>>>> -c                   Check for bad blocks and add them to the badblock list
>>>> -f                   Force checking even if filesystem is marked clean
>>>> -v                   Be verbose
>>>> -b superblock        Use alternative superblock
>>>> -B blocksize         Force blocksize when looking for superblock
>>>> -j external_journal  Set location of the external journal
>>>> -l bad_blocks_file   Add to badblocks list
>>>> -L bad_blocks_file   Set badblocks list
>>>> -z undo_file         Create an undo file
>>>> 
>>>> ## Depending on how you initially configured AstLinux, check if you also have a ASTKD partition
>>>> ## (typically /dev/sda3 if it exists)
>>>> runnix# findfs LABEL=ASTKD
>>>> ## If the ASTKD label exists repeat the e2fsck steps above using the ASTKD label partition.
>>>> 
>>>> ## reboot by issuing "exit"
>>>> runnix# exit
>>>> --
>>>> 
>>>> Lonnie
>>>> 
>>>> 
>>>> On Aug 29, 2017, at 7:02 AM, Tim Turpin <tt...@z-...> wrote:
>>>> 
>>>>> Never mind.  After sitting at that point for about 15 minutes, it somehow recovered and finished booting up.  All seems well now.
>>>>> 
>>>>> From: Tim Turpin [mailto:tt...@z-...]
>>>>> Sent: Tuesday, August 29, 2017 7:34 AM
>>>>> To: ast...@li...
>>>>> Subject: [Astlinux-users] ASTLinux stopped booting
>>>>> 
>>>>> We took a power hit on our test system yesterday, and now it will only load up to a certain point and stops with the following screen:
> 
> Lonnie, don't give up so quickly :-).
> What you have already is quite a good starting point.
> What about adding "something needed" to our initrd busybox.config …
> 
> Michael

The initrd busybox config is not the problem, this is a fundamental problem using shell, though easily solved with bash, but bash is not a practical option with our initrd.

Get exit status of process that's piped to another
https://unix.stackexchange.com/questions/14270/get-exit-status-of-process-thats-piped-to-another

I dont see an elegant solution.

My only other thought would be to create a background "spinner" process, then call "e2fsck -y $1 >/dev/null" and status="$?" then stop the background spinner process.

Lonnie

> Am 29.08.2017 um 20:12 schrieb Lonnie Abelbeck <li...@lo...>:
> 
> Hi Devs,
> 
> David made a good point, so I looked into piping the output of e2fsck to a "spinner" shell function, triggered by a newline and update no more often then ever second ...
> --
> spinner()
> {
>   local i sp secs nsecs line IFS
> 
>   i=0
>   secs=0
> 
>   unset IFS
>   while read line; do
>     nsecs=$(cut -d. -f1 /proc/uptime)
>     if [ $nsecs -ne $secs ]; then
>       secs=$nsecs
>       i=$(((i+1)%4))
>       case $i in
>         1) sp='-' ;;
>         2) sp='\' ;;
>         3) sp='|' ;;
>         *) sp='/' ;;
>       esac
>       echo -ne " Working[${sp}] \r"
>     fi
>   done
> 
>   echo -ne "            \r"
> }
> --
> It took some work getting this to work without printf, no date, no bash ${str:1:1}, etc., but it works with the current Busybox configs.
> 
> But ... here are the two places "spinner" would apply to ...
> 
> 1) in the initrd, linuxrc ...
> --
>     e2fsck -y $ASTURW >/dev/null
>     status=$?
> 
>     case $status in
> --
> 
> 2) in astlinux, /etc/rc ...
> --
>   echo "Checking $1"
>   e2fsck -y $1 >/dev/null
> 
>   status="$?"
> 
>   case $status in
> --
> 
> The plan would be to use:
> --
> e2fsck -y $1 | spinner
> --
> But the return value of $? is no longer from e2fsck, but rather spinner
> 
> You might try ...
> --
> ( e2fsck -y $1 ; status="$?" ) | spinner
> --
> Almost, but now "status" is defined in a subshell which is not propagated up and lost.
> 
> A simple but ugly solution is to use a temp file ...
> --
> ( e2fsck -y $1 ; echo "$?" > /tmp/e2fsck_status ) | spinner
> status="$(cat /tmp/e2fsck_status)"
> rm /tmp/e2fsck_status
> --
> but not to mention the initrd does not currently mount a /tmp filesystem. We would have to have to create and delete a tmpfs filesystem just to get the return value from e2fsck. :-(
> 
> I'm leaning toward leaving things as is, comments ?
> 
> Lonnie
> 
> 
> On Aug 29, 2017, at 8:20 AM, Lonnie Abelbeck <li...@lo...> wrote:
> 
>> Hi David,
>> 
>> Agreed, no e2fsck output when it is forced to work for a long period is not ideal.  Thankfully this is a very rare occurrence.
>> 
>> If we can come up with a clever spinning wheel or such counting the lines e2fsck generates might be a good addition.
>> 
>> Recall this all occurs in the "initrd", so the available packages and BusyBox config is quite limited.
>> 
>> Any further discussion on this idea should move to the astlinux-devel list.
>> 
>> Lonnie
>> 
>> 
>> On Aug 29, 2017, at 7:48 AM, David Kerr <Da...@Ke...> wrote:
>> 
>>> Is that a good idea (redirecting e2fsck to null)?  15 minutes is a long time to wait with no indication of anything happening.
>>> 
>>> David
>>> 
>>> On Tue, Aug 29, 2017 at 8:33 AM, Lonnie Abelbeck <li...@lo...> wrote:
>>> Hi Tim,
>>> 
>>> Yes, it was running "e2fsck" with stdout redirected to /dev/null so you did not see it working ... eventually e2fsck will return with a result code if left long enough.
>>> 
>>> Good to hear your filesystem is now clean, you might reboot once again to make sure the filesystem is good.
>>> 
>>> BTW, if the automatic e2fsck repair did not work, here is the manual fallback procedure:
>>> --
>>> ## reboot, and quickly when the RUNNIX boot menu appears type "shell"
>>> boot: shell
>>> 
>>> ## Wait for a "runnix# " CLI prompt.
>>> ## determine the first Linux ext2 partition, usually always /dev/sda2
>>> runnix# findfs LABEL=ASTURW
>>> 
>>> ## using the findfs result, run e2fsck manually, you may want to add -y or -p options
>>> runnix# e2fsck /dev/sda2
>>> 
>>> ## output a list of options for e2fsck
>>> runnix# e2fsck
>>> Usage: e2fsck [-panyrcdfktvDFV] [-b superblock] [-B blocksize]
>>>                [-l|-L bad_blocks_file] [-C fd] [-j external_journal]
>>>                [-E extended-options] [-z undo_file] device
>>> 
>>> Emergency help:
>>> -p                   Automatic repair (no questions)
>>> -n                   Make no changes to the filesystem
>>> -y                   Assume "yes" to all questions
>>> -c                   Check for bad blocks and add them to the badblock list
>>> -f                   Force checking even if filesystem is marked clean
>>> -v                   Be verbose
>>> -b superblock        Use alternative superblock
>>> -B blocksize         Force blocksize when looking for superblock
>>> -j external_journal  Set location of the external journal
>>> -l bad_blocks_file   Add to badblocks list
>>> -L bad_blocks_file   Set badblocks list
>>> -z undo_file         Create an undo file
>>> 
>>> ## Depending on how you initially configured AstLinux, check if you also have a ASTKD partition
>>> ## (typically /dev/sda3 if it exists)
>>> runnix# findfs LABEL=ASTKD
>>> ## If the ASTKD label exists repeat the e2fsck steps above using the ASTKD label partition.
>>> 
>>> ## reboot by issuing "exit"
>>> runnix# exit
>>> --
>>> 
>>> Lonnie
>>> 
>>> 
>>> On Aug 29, 2017, at 7:02 AM, Tim Turpin <tt...@z-...> wrote:
>>> 
>>>> Never mind.  After sitting at that point for about 15 minutes, it somehow recovered and finished booting up.  All seems well now.
>>>> 
>>>> From: Tim Turpin [mailto:tt...@z-...]
>>>> Sent: Tuesday, August 29, 2017 7:34 AM
>>>> To: ast...@li...
>>>> Subject: [Astlinux-users] ASTLinux stopped booting
>>>> 
>>>> We took a power hit on our test system yesterday, and now it will only load up to a certain point and stops with the following screen:

Lonnie, don't give up so quickly :-).
What you have already is quite a good starting point.
What about adding "something needed" to our initrd busybox.config …

Michael

http://www.mksolutions.info

Hi Devs,

David made a good point, so I looked into piping the output of e2fsck to a "spinner" shell function, triggered by a newline and update no more often then ever second ...
--
spinner()
{
  local i sp secs nsecs line IFS

  i=0
  secs=0

  unset IFS
  while read line; do
    nsecs=$(cut -d. -f1 /proc/uptime)
    if [ $nsecs -ne $secs ]; then
      secs=$nsecs
      i=$(((i+1)%4))
      case $i in
        1) sp='-' ;;
        2) sp='\' ;;
        3) sp='|' ;;
        *) sp='/' ;;
      esac
      echo -ne " Working[${sp}] \r"
    fi
  done

  echo -ne "            \r"
}
--
It took some work getting this to work without printf, no date, no bash ${str:1:1}, etc., but it works with the current Busybox configs.

But ... here are the two places "spinner" would apply to ...

1) in the initrd, linuxrc ...
--
    e2fsck -y $ASTURW >/dev/null
    status=$?

    case $status in
--

2) in astlinux, /etc/rc ...
--
  echo "Checking $1"
  e2fsck -y $1 >/dev/null

  status="$?"

  case $status in
--

The plan would be to use:
--
e2fsck -y $1 | spinner
--
But the return value of $? is no longer from e2fsck, but rather spinner

You might try ...
--
( e2fsck -y $1 ; status="$?" ) | spinner
--
Almost, but now "status" is defined in a subshell which is not propagated up and lost.

A simple but ugly solution is to use a temp file ...
--
( e2fsck -y $1 ; echo "$?" > /tmp/e2fsck_status ) | spinner
status="$(cat /tmp/e2fsck_status)"
rm /tmp/e2fsck_status
--
but not to mention the initrd does not currently mount a /tmp filesystem. We would have to have to create and delete a tmpfs filesystem just to get the return value from e2fsck. :-(

I'm leaning toward leaving things as is, comments ?

Lonnie


On Aug 29, 2017, at 8:20 AM, Lonnie Abelbeck <li...@lo...> wrote:

> Hi David,
> 
> Agreed, no e2fsck output when it is forced to work for a long period is not ideal.  Thankfully this is a very rare occurrence.
> 
> If we can come up with a clever spinning wheel or such counting the lines e2fsck generates might be a good addition.
> 
> Recall this all occurs in the "initrd", so the available packages and BusyBox config is quite limited.
> 
> Any further discussion on this idea should move to the astlinux-devel list.
> 
> Lonnie
> 
> 
> On Aug 29, 2017, at 7:48 AM, David Kerr <Da...@Ke...> wrote:
> 
>> Is that a good idea (redirecting e2fsck to null)?  15 minutes is a long time to wait with no indication of anything happening.
>> 
>> David
>> 
>> On Tue, Aug 29, 2017 at 8:33 AM, Lonnie Abelbeck <li...@lo...> wrote:
>> Hi Tim,
>> 
>> Yes, it was running "e2fsck" with stdout redirected to /dev/null so you did not see it working ... eventually e2fsck will return with a result code if left long enough.
>> 
>> Good to hear your filesystem is now clean, you might reboot once again to make sure the filesystem is good.
>> 
>> BTW, if the automatic e2fsck repair did not work, here is the manual fallback procedure:
>> --
>> ## reboot, and quickly when the RUNNIX boot menu appears type "shell"
>> boot: shell
>> 
>> ## Wait for a "runnix# " CLI prompt.
>> ## determine the first Linux ext2 partition, usually always /dev/sda2
>> runnix# findfs LABEL=ASTURW
>> 
>> ## using the findfs result, run e2fsck manually, you may want to add -y or -p options
>> runnix# e2fsck /dev/sda2
>> 
>> ## output a list of options for e2fsck
>> runnix# e2fsck
>> Usage: e2fsck [-panyrcdfktvDFV] [-b superblock] [-B blocksize]
>>                [-l|-L bad_blocks_file] [-C fd] [-j external_journal]
>>                [-E extended-options] [-z undo_file] device
>> 
>> Emergency help:
>> -p                   Automatic repair (no questions)
>> -n                   Make no changes to the filesystem
>> -y                   Assume "yes" to all questions
>> -c                   Check for bad blocks and add them to the badblock list
>> -f                   Force checking even if filesystem is marked clean
>> -v                   Be verbose
>> -b superblock        Use alternative superblock
>> -B blocksize         Force blocksize when looking for superblock
>> -j external_journal  Set location of the external journal
>> -l bad_blocks_file   Add to badblocks list
>> -L bad_blocks_file   Set badblocks list
>> -z undo_file         Create an undo file
>> 
>> ## Depending on how you initially configured AstLinux, check if you also have a ASTKD partition
>> ## (typically /dev/sda3 if it exists)
>> runnix# findfs LABEL=ASTKD
>> ## If the ASTKD label exists repeat the e2fsck steps above using the ASTKD label partition.
>> 
>> ## reboot by issuing "exit"
>> runnix# exit
>> --
>> 
>> Lonnie
>> 
>> 
>> On Aug 29, 2017, at 7:02 AM, Tim Turpin <tt...@z-...> wrote:
>> 
>>> Never mind.  After sitting at that point for about 15 minutes, it somehow recovered and finished booting up.  All seems well now.
>>> 
>>> From: Tim Turpin [mailto:tt...@z-...]
>>> Sent: Tuesday, August 29, 2017 7:34 AM
>>> To: ast...@li...
>>> Subject: [Astlinux-users] ASTLinux stopped booting
>>> 
>>> We took a power hit on our test system yesterday, and now it will only load up to a certain point and stops with the following screen:

Devs, (update)

> --
> Rule: Never remove a directory in the path '/oldroot/mnt/asturw/...'
> --

I still recommend the above rule.

Though looking at the unionfs kernel documentation

-=-=-=-= Documentation/filesystems/unionfs/usage.txt -=-=-=-=
CACHE CONSISTENCY
=================

If you modify any file on any of the lower branches directly, while there is
a Unionfs 2.x mounted above any of those branches, you should tell Unionfs
to purge its caches and re-get the objects.  To do that, you have to
increment the generation number of the superblock using the following
command:

# mount -t unionfs -o remount,incgen none MOUNTPOINT
-=-=-=-=

Indeed, if I remove a directory in the path '/oldroot/mnt/asturw/...' and *immediately* follow it with the command ...
--
mount -t unionfs -o remount,incgen none /
--
it seems to work (*much* better than not issuing the 'remount,incgen'), but I'm not convinced it is *perfect*.

BTW as before, directly removing a file in '/oldroot/mnt/asturw/...' does not require the 'remount,incgen' and continues to work fine, as it has in the past.  unionfs automatically handles the lower file removal case.

So, for development purposes only, I mention this 'remount,incgen' workaround for directly removing a directory in the lower filesystem while unionfs is mounted.  Best to also kernel-reboot or reboot afterwords.


BTW, in Linux kernels 3.18 and beyond the unionfs alternative is overlayfs, but overlayfs forbids any lower direct file manipulation while mounted ...

-=-=-=-= Documentation/filesystems/overlayfs.txt -=-=-=-=
Changes to underlying filesystems
---------------------------------

Offline changes, when the overlay is not mounted, are allowed to either
the upper or the lower trees.

Changes to the underlying filesystems while part of a mounted overlay
filesystem are not allowed.  If the underlying filesystem is changed,
the behavior of the overlay is undefined, though it will not result in
a crash or deadlock.
-=-=-=-=


Lonnie




On Jun 1, 2017, at 8:30 PM, Lonnie Abelbeck <li...@lo...> wrote:

> 
> On Jun 1, 2017, at 7:21 PM, David Kerr <Da...@Ke...> wrote:
> 
>> Okay, so...  occasionally I replace a file in e.g. /usr/sbin while testing some change I may have made.  When that change is eventually incorporated into the build tree I am in the habit of going into /oldroot/mnt/asturw/usr and then rm -rf sbin.
>> 
>> I think you are saying that this is bad? Right?
> 
> I used to do that as well, don't remove any directories, though removing a file is fine.  I found this out by cleaning-up after editing the traffic shaper plugin for testing.
> --
> Rule: Never remove a directory in the path '/oldroot/mnt/asturw/...'
> --
> 
> 
>> So instead I should go into /oldroot/mnt/asturw/usr/sbin and rm the file, and just leave the sbin directory alone?
> 
> Yes, to help, the new "show-union" will not show you the empty directories anymore.  Simply ignore the ASTURW empty directories.
> 
> Some day we may consider automatically cleaning up these empty directories if we ever thought they could be a problem, but a non-issue today.  That would have to be done in the initrd.
> 
> 
>> What are the consequences of erasing the directory?  And if I do erase a directory will a reboot make things okay again?
> 
> It seems any open file in the path will get flushed from memory, typically the AIF plugin helper scripts will be effected if '/oldroot/mnt/asturw/usr/...' and as such you won't get a clean shutdown, you will need to power cycle the box.
> 
> We can hope the unionfs folks will generate a new set of patches, but I would expect we just need to live with this.
> 
> As I recall, many years ago in the Linux 2.6 days. several versions of unionfs ago, this same problem existed.
> 
> Lonnie
> 
> 
> 
>> Thanks
>> David
>> 
>> On Thu, Jun 1, 2017 at 9:55 AM, Lonnie Abelbeck <li...@lo...> wrote:
>> Hi Devs,
>> 
>> First, it has been 10 days since we bumped the kernel to 3.16.43, and things appear to be running solid at this point, much thanks for all the assistance in getting this major task accomplished.
>> 
>> Though I ran across a glitch, with unionfs, basically you no longer can remove a *directory* in the path '/oldroot/mnt/asturw/...' as unionfs with kernel 3.16 has an issue with that.  Such a deletion now also effects open files in the same path, in memory.
>> 
>> Removing a *file* in the '/oldroot/mnt/asturw/...' path appears to be OK, which is important, since that is the only way to "undo" an edit of a file on '/oldroot/mnt/asturo/...'
>> 
>> I sent an email to Erez Zadok at Stony Brook University as well as the unionfs mailing list, where I go into more detail.
>> 
>> [Unionfs] unionfs with Linux 3.16
>> http://www.fsl.cs.sunysb.edu/pipermail/unionfs/2017-May/006196.html
>> 
>> I looked over our scripts and we only needed a few changes to implement the new rule:
>> --
>> Rule: Never remove a directory in the path '/oldroot/mnt/asturw/...'
>> --
>> (Revision: 8361)
>> 
>> The 'show-union' command now only shows files, so as not to encourage directory removal, except 'show-union all' shows directories as well.
>> 
>> Any stale directories on ASTURW should not happen for normal users in production, but during development it can be handy to edit ASTURO files, so if you wanted to remove these stale directories (leaving them be is fine) you need to reboot to RUNNIX and mount /dev/sda2 as -t ext2.
>> 
>> Lonnie
>> 
>> 
>> ------------------------------------------------------------------------------
>> Check out the vibrant tech community on one of the world's most
>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>> _______________________________________________
>> Astlinux-devel mailing list
>> Ast...@li...
>> https://lists.sourceforge.net/lists/listinfo/astlinux-devel
>> 
>> ------------------------------------------------------------------------------
>> Check out the vibrant tech community on one of the world's most
>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot_______________________________________________
>> Astlinux-devel mailing list
>> Ast...@li...
>> https://lists.sourceforge.net/lists/listinfo/astlinux-devel
> 
> 
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> Astlinux-devel mailing list
> Ast...@li...
> https://lists.sourceforge.net/lists/listinfo/astlinux-devel
> 
> 

On Aug 18, 2017, at 10:20 AM, Armin Tüting <arm...@tu...> wrote:

> On Thu, 2017-08-17 at 10:38 -0500, Lonnie Abelbeck wrote:
>> The AstLinux Project is now hosted on GitHub, both 'svn checkout' and
>> 'git clone' are supported: 
> Thanks Loonie.
> I've done a git clone and build - all works fine!
> The only thing is the versioning - I'm getting 'astlinux-1.3-05163a'.
> Looks a bit strange...
> 
> ...
> 
> Regards,
> Armin.

Hi Armin, thanks for taking the new GitHub repo for a spin.

Yes, if you use a "git clone" for the astlinux-project/astlinux.git and build within git, the "astlinux-1.3-05163a" release includes the current git SHA1 hash "05163a" (first 6 chars), with that you can show the last included commit:
--
https://github.com/astlinux-project/astlinux/commit/05163a
--

Or the history of the current and previous commits:
--
https://github.com/astlinux-project/astlinux/commits/05163a
--

You could also "svn co https://github.com/astlinux-project/astlinux.git/trunk" and build within an SVN checkout, the same version as above will be marked as "astlinux-1.3-3360-05163a" including a SVN revision number.

Your choice.

Lonnie

On Thu, 2017-08-17 at 10:38 -0500, Lonnie Abelbeck wrote:
> The AstLinux Project is now hosted on GitHub, both 'svn checkout' and
> 'git clone' are supported: 
Thanks Loonie.
I've done a git clone and build - all works fine!
The only thing is the versioning - I'm getting 'astlinux-1.3-05163a'.
Looks a bit strange...

...

Regards,
Armin.

Thanks Lonnie.  I think Github is a much better platform for astlinux and I
am honored to have Pull Request #1 accepted into the master.

David

On Thu, Aug 17, 2017 at 11:38 AM, Lonnie Abelbeck <li...@lo...
> wrote:

> The AstLinux Project is now hosted on GitHub, both 'svn checkout' and 'git
> clone' are supported:
>
> AstLinux Project at GitHub
> https://github.com/astlinux-project/astlinux
>
> All current development is on GitHub. Pull Requests are welcomed.
>
> Note: The Sourgeforge mailing lists [astlinux-users] and [astlinux-devel]
> will continue to be used as before.
>
> For AstLinux users, there should be no noticeable change.
>
> For AstLinux developers, custom builds will require a new checkout:
> --
> svn co https://github.com/astlinux-project/astlinux.git/trunk
> astlinux/trunk
> --
> The Developer docs have been updated, but not much has changed:
> https://doc.astlinux-project.org/devdoc:documentation
>
>
> Special thanks to David Kerr for assisting in researching and testing,
> making the migration straightforward.
>
> AstLinux Team
>
>
> ------------------------------------------------------------
> ------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> Astlinux-devel mailing list
> Ast...@li...
> https://lists.sourceforge.net/lists/listinfo/astlinux-devel
>

The AstLinux Project is now hosted on GitHub, both 'svn checkout' and 'git clone' are supported: 

AstLinux Project at GitHub
https://github.com/astlinux-project/astlinux

All current development is on GitHub. Pull Requests are welcomed. 

Note: The Sourgeforge mailing lists [astlinux-users] and [astlinux-devel] will continue to be used as before.

For AstLinux users, there should be no noticeable change.

For AstLinux developers, custom builds will require a new checkout:
--
svn co https://github.com/astlinux-project/astlinux.git/trunk astlinux/trunk
--
The Developer docs have been updated, but not much has changed:
https://doc.astlinux-project.org/devdoc:documentation


Special thanks to David Kerr for assisting in researching and testing, making the migration straightforward.

AstLinux Team

Announcing AstLinux Release: 1.3.0

More Info: AstLinux Project
http://www.astlinux-project.org/

AstLinux 1.3.0 Highlights:
• Major upgrade to Linux Kernel 3.16.44, including the RUNNIX bootloader
• The default serial baud rate is now 115200 instead of the previous 19200
• New firewall "net-prefix-translation" plugin, provides NPTv6 (Network Prefix Translation) for IPv6
• Updated firewall "traffic-shaper" plugin, use fq_codel (Fair Queueing CoDel) for both 'htb' and 'hfsc' types
• New command "acme-client" to generate Let's Encrypt certificates using the ACME protocol
• Added support for VMware Tools, vmw_pvscsi and virtio-scsi disk drivers (genx86_64-vm)
• Web Interface enhancements and package upgrades providing important security and bug fixes

Full ChangeLog:
http://svn.code.sf.net/p/astlinux/code/tags/1.3.0/docs/ChangeLog.txt

All users are encouraged to upgrade.

AstLinux Team

On Aug 13, 2017, at 12:24 PM, David Kerr <Da...@Ke...> wrote:

> Devs,
>   For a while now I have been pondering whether to advocate for moving Astlinux source control from Sourceforge to Github.  The reason is the power of collaborative development brought by Github (forks, pull requests, etc.) which will make it much easier to manage updates to AstLinux.
> 
> The problem of course is that our development processes and scripts are pretty tied to SVN, and GIT is a "different way of doing things"
> 
> But I thought I would run a test and see what happens.  So this weekend I used one of the migration tools (svn2git) and copied the entire AstLinux source over to github... branches, tags, commit history and all.  And I asked Lonnie to go kick the tires.
> 
> Github supports SVN as a client, so I was then able to svn checkout Astlinux and successfully ran a clean build.  This is critical because for at least some transition period Astlinux will still assume SVN in its build environment -- but over time it is possible we could change that.
> 
> But.  And of course there is always a but.  We have run into an issue.  For some reason the revision number is out-of-sync.  So for example on Sourceforge astlinux branch 1.0 is at 8481, the exact equivalent on Github thinks it is at revision 8467.  We're not sure why.
> 
> Also, Github assumes that the current dev code base is in "master" branch.  Sourceforge equivalent is "trunk" and that is migrated cleanly.  But, another but, Astlinux on Sourceforge is using Branches/1.0 as the current dev code base and trunk is wildly out of date.  That needs to get resolved either on Sourceforge prior to importing to Github, or after importing.
> 
> Actions:
> 1) Please respond with your thoughts.  Do you see a problem if we move to GitHub?
> 2) Is there a reason "trunk" is so out-of date?  Is it okay if we merge branches/1.0 into that on Sourceforge or is it better to leave Sourceforge in the state it is (as a permanent record) and do the merge once we are on GitHub.
> 3) There are a lot of branches.  We should clean these up... delete anything not needed.  Again decision to be made as to whether to do this on Sourceforge prior to migration, or do the cleanup on Github. Please identify any branches that you think should NOT be deleted.
> 
> Feel free to take a look and kick the tires at https://github.com/dkerr64/astlinux but note that when we pull the trigger we will host it at https://github.com/astlinux-project so any changes made in my github account will get lost.
> 
> Personally I vote to do the migration. With 1.3 just tagged, now might be a good time.
> 
> Thanks
> David

I'm old and don't like change, but GitHub is a dream to work on compared to Sourceforge.  For what we do SVN is far easier to use than git, but per David's proof-of-concept the SVN wrapper around GitHub's git seems to work.  Regardless I would expect some problems, most likely fixable.  I think we figured out the revision number sync issue, seems to work after a fresh SVN commit.

I also worry about Sourceforge's future.  Though we would keep our mailing lists on Sourceforge as GitHub has no equivalent.  GitHub "Issues" are not the same.

I'm on the fence here, what we have works and any transition has pitfalls.  Darrick what do you think ?


If we did this ...
Before the transition we should remove all the old unused branches, keep 0.4 to 0.7 and rename 1.0 to trunk.

Darrick, what is your GitHub username "dhartman" I presume ?  Off-list I would need your associated GitHub account email address.

Lonnie

Devs,
  For a while now I have been pondering whether to advocate for moving
Astlinux source control from Sourceforge to Github.  The reason is the
power of collaborative development brought by Github (forks, pull requests,
etc.) which will make it much easier to manage updates to AstLinux.

The problem of course is that our development processes and scripts are
pretty tied to SVN, and GIT is a "different way of doing things"

But I thought I would run a test and see what happens.  So this weekend I
used one of the migration tools (svn2git) and copied the entire AstLinux
source over to github... branches, tags, commit history and all.  And I
asked Lonnie to go kick the tires.

Github supports SVN as a client, so I was then able to svn checkout
Astlinux and successfully ran a clean build.  This is critical because for
at least some transition period Astlinux will still assume SVN in its build
environment -- but over time it is possible we could change that.

But.  And of course there is always a but.  We have run into an issue.  For
some reason the revision number is out-of-sync.  So for example on
Sourceforge astlinux branch 1.0 is at 8481, the exact equivalent on Github
thinks it is at revision 8467.  We're not sure why.

Also, Github assumes that the current dev code base is in "master" branch.
Sourceforge equivalent is "trunk" and that is migrated cleanly.  But,
another but, Astlinux on Sourceforge is using Branches/1.0 as the current
dev code base and trunk is wildly out of date.  That needs to get resolved
either on Sourceforge prior to importing to Github, or after importing.

Actions:
1) Please respond with your thoughts.  Do you see a problem if we move to
GitHub?
2) Is there a reason "trunk" is so out-of date?  Is it okay if we merge
branches/1.0 into that on Sourceforge or is it better to leave Sourceforge
in the state it is (as a permanent record) and do the merge once we are on
GitHub.
3) There are a lot of branches.  We should clean these up... delete
anything not needed.  Again decision to be made as to whether to do this on
Sourceforge prior to migration, or do the cleanup on Github. Please
identify any branches that you think should NOT be deleted.

Feel free to take a look and kick the tires at
https://github.com/dkerr64/astlinux but note that when we pull the trigger
we will host it at https://github.com/astlinux-project so any changes made
in my github account will get lost.

Personally I vote to do the migration. With 1.3 just tagged, now might be a
good time.

Thanks
David