Discussion:
Feature request...?
(too old to reply)
Andre Grueneberg
2012-08-06 14:15:06 UTC
Permalink
Hi Nick

Nick Andre schrieb:

NA> Is it possible for BinkD to "busy out" and refuse ALL incoming
NA> connections by setting a semaphore file of some kind?

What do you mean by "busy out"? Currently there is no such functionality --
well, I believe you could do some of it using integrated Perl though.

What about ongoing connections?

What's the rationale behind this anyway? So far I don't see the point what
you'd like to accomplish? I'd say, you could stop the daemon and start it again
when your whatever-maintenance is over?!

CU Andre E-Mail: ***@grueneberg.de
Nick Andre
2012-08-06 10:16:42 UTC
Permalink
On 06 Aug 12 18:15:06, Andre Grueneberg said the following to Nick Andre:

AG> NA> Is it possible for BinkD to "busy out" and refuse ALL incoming
AG> NA> connections by setting a semaphore file of some kind?
AG>
AG> What do you mean by "busy out"? Currently there is no such functionality --
AG> well, I believe you could do some of it using integrated Perl though.

For example, when the "max servers" setting is exceeded, any further incoming
connections are denied and the remote system returns a "Busy" status (Irex)

AG> What about ongoing connections?

Ongoing connections would continue as usual until they are finished.

AG> What's the rationale behind this anyway? So far I don't see the point what
AG> you'd like to accomplish? I'd say, you could stop the daemon and start it
AG> again when your whatever-maintenance is over?!

No I do not want to stop the daemon as that may cause problems for systems
who connect again on a high traffic server system which moves a lot of mail. I
want to busy-out any new incoming connections and prevent any further ongoing
ones.

Again - just wondering if its possible to do this and/or add this.

Nick
mark lewis
2012-08-11 13:23:06 UTC
Permalink
NA> Is it possible for BinkD to "busy out" and refuse ALL incoming
NA> connections by setting a semaphore file of some kind?

AG> What do you mean by "busy out"?

like a POTS modem does by taking the phone offhook... effectively the
line/connection is busy and no one can connect...

AG> Currently there is no such functionality -- well, I believe you
AG> could do some of it using integrated Perl though.

IF perl is (still) in one's binkd, how would this be done?

AG> What about ongoing connections?

they continue until they finish and then binkd would block... it would be good
for it to also create a semaphore signalling that it has blocked all inbound
connections...

AG> What's the rationale behind this anyway? So far I don't see the
AG> point what you'd like to accomplish?

as i mentioned before, when i was looking for such a thing, in my case it was
for log file processing...

AG> I'd say, you could stop the daemon and start it again when your
AG> whatever-maintenance is over?!

how is this to be done? i know of no automated way of doing it...

)\/(ark
Andre Grueneberg
2012-08-12 18:23:14 UTC
Permalink
Hi mark

mark lewis schrieb:

AG>> Currently there is no such functionality -- well, I believe you
AG>> could do some of it using integrated Perl though.
ml> IF perl is (still) in one's binkd, how would this be done?

I'd take a look at implementing an on_handshake trigger which drops the
connection.

AG>> What about ongoing connections?
ml> they continue until they finish and then binkd would block... it
ml> would be good for it to also create a semaphore signalling that it
ml> has blocked all inbound connections...

And what about outgoing connections?

AG>> What's the rationale behind this anyway? So far I don't see the
AG>> point what you'd like to accomplish?
ml> as i mentioned before, when i was looking for such a thing, in my
ml> case it was for log file processing...

Under Unix that's not much of an issue, but IIRC it will require a restart
after rotating.

AG>> I'd say, you could stop the daemon and start it again when your
AG>> whatever-maintenance is over?!
ml> how is this to be done? i know of no automated way of doing it...

As you're not indicating the OS: /etc/init.d/binkd stop; /etc/init.d/binkd
start. :-> Under Windows you're most likely running binkd as a service ... net
stop and start should do the trick.

CU Andre E-Mail: ***@grueneberg.de
mark lewis
2012-08-13 08:44:18 UTC
Permalink
AG>> Currently there is no such functionality -- well, I believe you
AG>> could do some of it using integrated Perl though.
ml> IF perl is (still) in one's binkd, how would this be done?

AG> I'd take a look at implementing an on_handshake trigger which drops
AG> the connection.

i don't know anything about that...

AG>> What about ongoing connections?
ml> they continue until they finish and then binkd would block... it
ml> would be good for it to also create a semaphore signalling that it
ml> has blocked all inbound connections...

AG> And what about outgoing connections?

if they are ongoing, let them finish... new ones are stopped until the
semaphore is removed...

AG>> What's the rationale behind this anyway? So far I don't see the
AG>> point what you'd like to accomplish?
ml> as i mentioned before, when i was looking for such a thing, in my
ml> case it was for log file processing...

AG> Under Unix that's not much of an issue, but IIRC it will require a
AG> restart after rotating.

i don't run binkd under linux but instead on OS/2... it was Warp 3 Connect but
i've recently been yanked kicking and screaming into the more modern world and
the system is now running on eCS 2.1... when i was first testing this, i found
that i could simply yank the file from under binkd and it would recreate it and
carry on... no problem... except for the following...

if i rename the file from under binkd and then process it for last months data,
if there's a new connection that comes in, i can loose data when i try to put
the current month's data back in place for binkd to continue writing to... my
grep tool locks the file (and binkd crys about that and data is lost [not
written] to the log) during processing... what i have to do right now is
something like the following...

1. manually terminate binkd. (would be MUCH better to exit w/ semaphore)
2. grep out the data for the specified month.

(set %dy to desired month)
(%2 is the four digit year)
(%mon_num is the two digit month number)
grep -h -e "^. .. %dy ..:..:.. \[.*\]" bdlog.bnd > bd%2%mon_num.bnd

3. grep out the data for the current month using the above.
4. copy or rename the current month's data back to the default binkd
log so that it only contains the current month's data.

there are months that i don't remember to do this... i recently had to do the
above for 2 years of data... that's a huge log file to go thru... one thing
that would REALLY be better would be the ability to define the log file name
something like

log x:\\binkd\\bd%YYYY%mm.log

bd201208.log

and let binkd handle closing the old one and opening the new one when the
time/dat rolls over... IIRC, Apache has this ability but Apache keeps the logs
open and that's not a good thing at times... the above is for monthly logs but
some folk would rather daily logs... use the same method but with whatever
"normal" indicators are used... POSIX stuff? fstr? i don't remember but i now
use it in some of my pascal code ;)

AG>> I'd say, you could stop the daemon and start it again when your
AG>> whatever-maintenance is over?!
ml> how is this to be done? i know of no automated way of doing it...

AG> As you're not indicating the OS: /etc/init.d/binkd stop;
AG> /etc/init.d/binkd start. :-> Under Windows you're most likely
AG> running binkd as a service ... net stop and start should do the
AG> trick.

aren't those simply scripts that send some special code to binkd? IME with
*nix, /etc/init.d or /etc/rc.d stuff is scripts that might send a signal to the
process or they might have some other way of terminating the process... again,
i know of no such method that is documented in the binkd docs...

another example is that i do have EMX here and can send *nix signals to
processes BUT they must be compiled with EMX for the signals to work... the
watcom compiles won't respond to EMX signals...

i just don't know why it has to be so hard to code for the use of disk-based
semaphore files no matter what OS is being used... sure, the other stuff may be
"more native" but the disk-based stuff should work all the time no matter
what...

bdexit.XXX where XXX is the errorlevel to exit with.
bdfreeze binkd creates binkdfrozen when connections
are prevented and all active connections
are complete.
bdfreeze.in binkd creates binkdfrozen.in when inbound
connections are prevented and all active
inbound connections are complete.
bdfreeze.out binkd creates binkdfrozen.in when outbound
connections are prevented and all active
outbound connections are complete.

when the freeze semaphore disappears, binkd
wakes up and goes back to normal processing.

define a semaphore directory for these to be placed in or i guess continue to
use the outbound directory... but that might get messy in a multiple network 5D
configuration... a dedicated semaphore directory would be best... i used to
keep my semaphores like this on a RAM disk so they weren't left over on reboot
but today i have specific cleanup routines in my boot sequence...

FWIW: in the above, the exit one would easily serve my purposes because my
script can catch the errorlevel and then loop waiting on another semaphore to
appear or go away depending on what i do and how i write it ;)

for others, the freeze ones might be beneficial... FrontDoor has numerous
semaphore files that it can use and some of the above ideas are taken from
there... with the freeze semaphore, all files are closed and then fully
reloaded after the freeze semaphore is removed... in other words, it is just
like an exit except that the program doesn't exit... this is a good thing for
those who have "tasklist" windows where they load programs in a certain order
and don't want them moved from that position in the list ;)

)\/(ark
Andre Grueneberg
2012-08-14 17:59:02 UTC
Permalink
Hi mark

mark lewis schrieb:

AG>>> Currently there is no such functionality -- well, I believe you
AG>>> could do some of it using integrated Perl though.
ml>> IF perl is (still) in one's binkd, how would this be done?
AG>> I'd take a look at implementing an on_handshake trigger which drops
AG>> the connection.
ml> i don't know anything about that...

I guess, you'll have to figure out the rest then. :/

AG>>> What about ongoing connections?
ml>> they continue until they finish and then binkd would block... it
ml>> would be good for it to also create a semaphore signalling that it
ml>> has blocked all inbound connections...
AG>> And what about outgoing connections?
ml> if they are ongoing, let them finish... new ones are stopped until
ml> the semaphore is removed...

But you're aware that this state doesn't mean that the logfile is not being
used anymore?
So far my understanding of your objective is to rotate log files ...

AG>> Under Unix that's not much of an issue, but IIRC it will require a
AG>> restart after rotating.
ml> i don't run binkd under linux but instead on OS/2... it was Warp 3
ml> Connect but i've recently been yanked kicking and screaming into
ml> the more modern world and the system is now running on eCS 2.1...

... and you're using OS/2 which basically means you'll need to shutdown binkd
completely to be able to replace the logfile. Otherwise you might end up in a
locking violation. Some process might be writing to it.

ml> 2. grep out the data for the specified month.
ml> 3. grep out the data for the current month using the above.

... and why do you need exact month boundaries? If you really want that, you'll
need to kill all binkd processes just before midnight of the month end ... in
order to make sure no process is still writing to the old file.

ml> log x:\\binkd\\bd%YYYY%mm.log

That indeed sounds like a feature request. I wonder if all target platforms
support strftime() ... at least C89 mandates it.

ml> and let binkd handle closing the old one and opening the new one
ml> when the time/dat rolls over... IIRC, Apache has this ability but
ml> Apache keeps the logs open and that's not a good thing at times...

To be honest, the way binkd handles logfiles is not optimal ... in a
multi-processing environment. If two processes open the logfile at the same
time, they might overwrite each others log line ... or some other race
conditions.

AG>> As you're not indicating the OS: /etc/init.d/binkd stop;
AG>> /etc/init.d/binkd start. :-> Under Windows you're most likely
AG>> running binkd as a service ... net stop and start should do the
AG>> trick.
ml> aren't those simply scripts that send some special code to binkd?

Not necessarily. But usually they are sending a SIGTERM to the process ... one
could also kill the process the hard way ...

ml> IME with *nix, /etc/init.d or /etc/rc.d stuff is scripts that might
ml> send a signal to the process or they might have some other way of
ml> terminating the process... again, i know of no such method that is
ml> documented in the binkd docs...

What do OS functionalities have to do with binkd?

ml> another example is that i do have EMX here and can send *nix
ml> signals to processes BUT they must be compiled with EMX for the
ml> signals to work... the watcom compiles won't respond to EMX
ml> signals...

But on OS/2 DosKillProcess() will do what you want ... all you need is some
command line tool which finds the relevant processes and kills those. At least
I find several such tools on
<http://www.os2site.com/sw/util/process/index.html>.

ml> i just don't know why it has to be so hard to code for the use of
ml> disk-based semaphore files no matter what OS is being used...

Because disk based semaphore files is a concept mostly known in DOS-like OSs.
Binkd originally rather resembles the Unix tradition of daemons ... i.e. having
multiple processes handling connections independent from each other, w/o static
configuration of the number of parallel processes.

ml> for others, the freeze ones might be beneficial... FrontDoor has
ml> numerous semaphore files

... because it's targeted to a DOS environment with each process running in a
dedicated (virtual) machine ... all the time.

If you need DOS behaviour, you'll need to wrap binkd into it ... alternatively
you may contribute code.

I might have a look into logfile name expansion using strftime() ...

CU Andre E-Mail: ***@grueneberg.de
mark lewis
2012-08-14 15:27:48 UTC
Permalink
AG>>> Currently there is no such functionality -- well, I believe you
AG>>> could do some of it using integrated Perl though.
ml>> IF perl is (still) in one's binkd, how would this be done?
AG>> I'd take a look at implementing an on_handshake trigger which drops
AG>> the connection.
ml> i don't know anything about that...

AG> I guess, you'll have to figure out the rest then. :/

sorry, but not... i'll be using a binkd that doesn't have perl stuffings in it
so they won't do me any good anyway ;)

AG>>> What about ongoing connections?
ml>> they continue until they finish and then binkd would block... it
ml>> would be good for it to also create a semaphore signalling that it
ml>> has blocked all inbound connections...
AG>> And what about outgoing connections?
ml> if they are ongoing, let them finish... new ones are stopped until
ml> the semaphore is removed...

AG> But you're aware that this state doesn't mean that the logfile is
AG> not being used anymore?

i noted in my previous post that binkd could indicate that it has closed all
files by creating a semaphore file in response to the one created to tell it to
freeze... having a semaphore to exit is one thing but i don't know that i want
binkd to just dive to the floor without finishing the current transfers...
perhaps binkd could create a BDALIVE semaphore and remove it when it exits...
this is another way that external scripts and programs could tell if binkd was
running... there might need to be two others indicating that the client is
active and one for the server being active... i'm thinking these would be
created and maintained by the main thread...

AG> So far my understanding of your objective is to rotate log files
AG> ...

in /my/ case, that is the _current_ objective but others have their reasons...
i can think of other possibilities but they are not worth discussing if these
simple items cannot be acquired :/

AG>> Under Unix that's not much of an issue, but IIRC it will require a
AG>> restart after rotating.
ml> i don't run binkd under linux but instead on OS/2... it was Warp 3
ml> Connect but i've recently been yanked kicking and screaming into
ml> the more modern world and the system is now running on eCS 2.1...

AG> ... and you're using OS/2 which basically means you'll need to
AG> shutdown binkd completely to be able to replace the logfile.
AG> Otherwise you might end up in a locking violation. Some process
AG> might be writing to it.

i've already explained that i can easily rename the log file out from under
binkd... the problem is not that... the problem is if binkd has connections
while the logfile is away and it creates a new one... now how to blend them
back together and in place for binkd to continue using?

ml> 2. grep out the data for the specified month.
ml> 3. grep out the data for the current month using the above.

AG> ... and why do you need exact month boundaries? If you really want

because i want monthly logs for one simple thing... that and monthly stats on
those logs (which is what started all of this mess for me years ago)...

AG> that, you'll need to kill all binkd processes just before midnight
AG> of the month end ... in order to make sure no process is still
AG> writing to the old file.

again... HOW! there is only the MANUAL method of hitting CTRL-C that i know of
and i do not plan on being up at the end of the month to do this manually...
nor do i plan on being up every night if i decide to do this nightly (which is
another aspect that i'm looking at for statistics)...

ml> log x:\\binkd\\bd%YYYY%mm.log

AG> That indeed sounds like a feature request. I wonder if all target
AG> platforms support strftime() ... at least C89 mandates it.

i know i've specifically mentioned if before but i'd have to go digging back in
my message base to see if i still have it... if it was over 2 years ago, i'm
sure i do not have it...

ml> and let binkd handle closing the old one and opening the new one
ml> when the time/dat rolls over... IIRC, Apache has this ability but
ml> Apache keeps the logs open and that's not a good thing at times...

AG> To be honest, the way binkd handles logfiles is not optimal ... in
AG> a multi-processing environment. If two processes open the logfile
AG> at the same time, they might overwrite each others log line ... or
AG> some other race conditions.

exactly... this is why i previously mentioned a logging routine that all
threads talked to to let /it/ do the logging and they just keep doing their
thing ;)

AG>> As you're not indicating the OS: /etc/init.d/binkd stop;
AG>> /etc/init.d/binkd start. :-> Under Windows you're most likely
AG>> running binkd as a service ... net stop and start should do the
AG>> trick.
ml> aren't those simply scripts that send some special code to binkd?

AG> Not necessarily. But usually they are sending a SIGTERM to the
AG> process ... one could also kill the process the hard way ...

the point of the question was that on *NIX those are scripts that do the heavy
lifting of sending the signal or whatever... i don't use windows for mission
critical operations so how to run binkd as a service is beyond me... but IF i
were to do such, i would NOT run it as a service because i want to _see_ it and
what it is doing ;)

ml> IME with *nix, /etc/init.d or /etc/rc.d stuff is scripts that might
ml> send a signal to the process or they might have some other way of
ml> terminating the process... again, i know of no such method that is
ml> documented in the binkd docs...

AG> What do OS functionalities have to do with binkd?

the OS provides the signaling capabilities, does it not? it seems to me that it
does in some cases...

ml> another example is that i do have EMX here and can send *nix
ml> signals to processes BUT they must be compiled with EMX for the
ml> signals to work... the watcom compiles won't respond to EMX
ml> signals...

AG> But on OS/2 DosKillProcess() will do what you want ... all you need
AG> is some command line tool which finds the relevant processes and
AG> kills those. At least I find several such tools on
AG> <http://www.os2site.com/sw/util/process/index.html>.

thanks for the link... i may take a look but this would be SOOOOOO much easier
if we could just create a simple disk-based semaphore file and the watch for
one to appear (BDFROZEN) or disappear (BDALIVE)...

IF binkd reacted to disk-based sepamhores, and if it knew called bdexit.15 was
for exiting with errorlevel 15, then we might have something like this in the
binkd controller script...

@echo off
rem set the screen to 120 columns and 50 lines
mode 120 50
cdd x:\binkd
:loop1
binkd2e -C binkd.cfg
if errorlevel EQ 15 goto logmaint
if errorlevel EQ 3 goto loop1
goto end
:logmaint
sleep 5
if not exist x:\path\to\semaphores\foo.bar goto loop1
goto loop2
:end
echo binkd controller script terminating...

then in our processing script, we might have something like this...

@echo off
rem let's set our we're alive semaphore before continuing
if exist x:\path\to\semaphores\foo.bar goto running
echo.>x:\path\to\semaphores\foo.bar
rem let's kill binkd before tearing the logs to shreds
echo.>x:\path\to\semaphores\bdexit.15
:loop
sleep 5
if not exist x:\path\to\semaphores\bdalive.* goto good
echo waiting on binkd to quit...
goto loop
:good
rem now that binkd is not running, let's get those logs analyzed
[analysis code here]
rem now let's start binkd back up by removing our semaphore
del x:\path\to\semaphore\foo.bar
goto end
:running
echo we're already running... good bye!
:end
echo %0 terminating...


or something like that... i don't recall the errorlevels that binkd exits
with... i can't remember what that errorlevel 3 is for nor what a normal CTRL-C
exit uses... the real controller script would be more detailed and check
numerous errorlevels on exit... the above is for example but it is very similar
to how i control many aspects of my bbs system's software... i can tell FD to
do almost anything simply by having it exit with a certain errorlevel and
catching that in my controller script... if binkd had this capability, it would
be extremely nice and much easier to control in an automated fashion ;)

ml> i just don't know why it has to be so hard to code for the use of
ml> disk-based semaphore files no matter what OS is being used...

AG> Because disk based semaphore files is a concept mostly known in
AG> DOS-like OSs. Binkd originally rather resembles the Unix tradition
AG> of daemons ... i.e. having multiple processes handling connections
AG> independent from each other, w/o static configuration of the number
AG> of parallel processes.

guess what? all of my servers on my *nix boxes also work with disk-based
semaphores ;) some of them are done with scripts that send signals and others
use other means BUT the thing is that those signals and semahores are know and
documented... i've not found and good documentation for binkd on OS/2, windows
or *nix that explains these things...

ml> for others, the freeze ones might be beneficial... FrontDoor has
ml> numerous semaphore files

AG> ... because it's targeted to a DOS environment with each process
AG> running in a dedicated (virtual) machine ... all the time.

no... frontdoor has numerous semaphores for external and automated control...
NOT because it is DOS based... whoever told you that was quite incorrect...

AG> If you need DOS behaviour, you'll need to wrap binkd into it ...

i don't need *DOS* behavoir... others and myself want external control
capabilities so we can control binkd easier and without manual intervention...

AG> alternatively you may contribute code.

i've already stated that i do not code C or C++... thanks but no thanks... i'll
pull the trigger of the gun pointed at my foot when /i/ want to :)

AG> I might have a look into logfile name expansion using strftime()
AG> ...

thanks... that would be one big help... at least for me in my goal of
processing the log files anyway... i really hated having to process the one i
had that was some 40 or 50 megs in size and covered several years... that's why
i came up with the grep thing i showed earlier... at least i was able to slice
and dice the logs into months and then analyze them... i don't want to do that
again and for now i try to remember to come in every few weeks and run the
process but this may not always be possible... having binkd do the filenaming
would be great! :)

)\/(ark
Mvan Le
2012-09-03 00:47:30 UTC
Permalink
ml> as i mentioned before, when i was looking for such a
ml> thing, in my case it was for log file processing...

AG> I'd say, you could stop the daemon and start it again when your
AG> whatever-maintenance is over?!

ml> how is this to be done? i know of no automated way of doing it...

Just insert a firewall rule to block the BinkD port.
mark lewis
2012-09-03 08:38:14 UTC
Permalink
ml> as i mentioned before, when i was looking for such a
ml> thing, in my case it was for log file processing...

AG> I'd say, you could stop the daemon and start it again when your
AG> whatever-maintenance is over?!

ml> how is this to be done? i know of no automated way of doing it...

MvanL> Just insert a firewall rule to block the BinkD port.

can't do that on this setup... this isn't a *nix box with iptables or
similar...

)\/(ark
Benny Pedersen
2012-09-10 00:46:20 UTC
Permalink
Hello mark!

03 Sep 2012 12:38, mark lewis wrote to Mvan Le:

ml>> as i mentioned before, when i was looking for such a
ml>> thing, in my case it was for log file processing...

AG>> I'd say, you could stop the daemon and start it again when your
AG>> whatever-maintenance is over?!

ml>> how is this to be done? i know of no automated way of doing it...

MvanL>> Just insert a firewall rule to block the BinkD port.

ml> can't do that on this setup... this isn't a *nix box with iptables or
ml> similar...

unplug rj45 cable so ?

hope its not wireless :=)


Regards Benny

... there can only be one way of life, and it works :)
mark lewis
2012-09-10 04:14:33 UTC
Permalink
ml>> as i mentioned before, when i was looking for such a
ml>> thing, in my case it was for log file processing...

AG>> I'd say, you could stop the daemon and start it again when your
AG>> whatever-maintenance is over?!

ml>> how is this to be done? i know of no automated way of doing it...

MvanL>> Just insert a firewall rule to block the BinkD port.

ml> can't do that on this setup... this isn't a *nix box with iptables or
ml> similar...

BP> unplug rj45 cable so ?

what part of "automated" is being missed in this discussion? :/

BP> hope its not wireless :=)

it isn't...

)\/(ark

Loading...