Re: Reflector Scanner Web Pages

Jason Williams (streak@ccwf.cc.utexas.edu)
Wed, 29 Oct 1997 23:41:48 -0600 (CST)


On Wed, 29 Oct 1997, Bill Woodland (Squeek) wrote:
> At 06:09 AM 10/28/97 -0600, Jason Williams wrote:
> >I've thought about adding a lot more fields.. min/max send and receive
> >rates required by the reflector, email addresses, rating fields, etc.
> >The problem in my aspect is it reduces the likelihood of more reflectors
> >being added to the scanner.
>
> Jason:
> Do it! Just make those other fields OPTIONAL. That way, if a LAZY ref
> operator (like me :) goes to fill it in, all they really have to give you
> is the name of the ref and the IP/hostname.

Yeah..I could do that... Right now none of the fields are optional except
URL. I'm all for adding in the rates..but as for ID.
The problem I've strugged with before is people entering in private
reflectors. I can see it now..some total loser adding in the IP of a
private reflector that has a private ID associated with it and enters the
ID as well. I've attempted to make things as automated as possible, so I
don't screen reflectors BEFORE they are physically added to the scanner.
I do tend to check the added reflectors within a day or two to make sure
they aren't private. I now also have a list of IPs of private reflectors
which CAN'T be added (up to 70 IPs right now in that list).

I'd like to possibly add ID support later..ie: manually going thru the
list of added refs and determining which IDs are public,etc...but that's a
major pain.

Related to that is the shere amount of work to dig up send/receive rates
on all the CURRENT reflectors..not an easy task (unless their telnet port
on the WP ref is wide open) and also very time consuming. It took me
about 2-3 weeks to go through the list and dig up URLs on the refs
(finding URLs specific to a ref can be hard too..especially with a lot of
Japanese refs when I can't read the pages at all)

I'd also like to include an email address of the operator..but many times
that's almost impossible to find out.

> I have a submit section on my
> web page, and none of the fields are actually required.

Yep..I know...I've seen a few people randomly hitting submit with empty
forms :)

> As you know, I
> modeled it after the Cornell phonebook entry, so it has all of the fields
> that are included in the phonebook, along with a few others:
>
> Administrator's Email address

not hard for new refs..a pain to find out for existing refs

> IP address or HostName

I've thought about adding hostname support...just think it'd screw up some
things..and checking for valid hostnames is a bit tougher for me.

> Conference ID

See above...private refs being entered becomes a possibility..and I don't
care to get another email from White Pine about a pissed off reflector
operator :)

> Is this a Lurker Only Reflector?
> Receive rate Minimum? Maximum?
> Transmit rate Minimum? Maximum?

All of those are cool..but are they really needed? rates are I know..but
the other stuff is just useful information about the ref.
also, it's hard to verify accurate information and keep the automation.

> Rating

I like your rating system..unfortunately for some refs, it varies
depending on time of day. Take Farago for example. It's g-rated during
office hours and unrated outsideof office hours. A lot of orignally
intended g-rated reflectors have turned R/X-rated simply because of lack
of control by the reflector operator. Do we put it's originally intended
rating or the actual rating?

Ratings in general are kind of subjective...Some reflectors are obvious
with the MOTD stating it's an adult reflector, g-rated reflector, etc.
Other reflectors aren't so obvious. Do I want reflector operators pissed
because I listed their ref as adult when it really isn't? Granted, it's
easy for me to change, but it also becomes a hassle. From experience, it
seems a lot of people seek out only the adult refs. Adding a rating like
this would simplify the search.

> Location
> City
> State
> Country

This wouldn't be too hard to add and figure out for most refs...but a real
pain for those non-US refs. I've used locations of refs to determine
relative distance (network-wise) to the ref to minimize packet loss and #
of hops to the ref. I've figured in most cases, the number of hops and
routers in between myself and refs in the US wouldn't be very bad at all.
That's the main reason why I only list Country.

You also have a listing for other comments...my comments section is quite
limited. I've thought about extending it with a message board for each
ref in the list or something. That way people could comment on each ref
individually and wouldn't have to rely on my short comments section alone.
It also lets others comment instead of just the one who added the
reflector.

But that's a future project...when I have gobs and gobs of spare time :)

> Let's all urge Streak to write the code
> necessary to take that input and properly populate his database.
>
> Jason:
> HINT HINT :)

Yeah :) The code itself isn't hard to implement...just more fields. The
hard part is going over each reflector (out of the 160-170 active ones)
and finding out all this information.

You also have the problem of superflous information...while a lot of that
information may be nice to have, it's not essential in the operation of CU
or to seek out reflectors. If I added all of that (and that would be a
TON of work for the data gathering), would people really look at it? A
lot of the download formats don't accept comments very well. The only
really useful fields I could add at this point are the ID and send/receive
rates. Those are used in the 0.92b2 PhoneBook and also in auto-connecting
to refs.

The database of refs is also growing..the internal format for the ref list
is about 500 lines... I don't have an unlimited amount of space on the
system where all of this is at, so I have to watch what all I do.

There's also a problem of how to print all this information out in a
concise form on the scanner web page itself. That's a TON of information
that needs to be added. Once it's there, presumably you'd want it
displayed. The web page is already fairly lengthy (140K for all the
refs). All the new fields would increase each reflector to 4-5 rows in a
table instead of just two. That's a lot of information to wade through
just to find a decent list of refs. One of the goals I had in designing
it was the philosophy of KISS (Keep it simple and small)..or as
small as possible. Having all that information there (assuming it doesn't
take months to find out all the information) can really clutter the
screen. I'm sure I could find some way to do it...just a matter of
playing around enough.

Now if White Pine would just get the ID and rates into the phonebook
format so you can import rates for each ref, I'd have more to play with
than just the Cornell version for importing rates.

In short...I'm all for adding in at least the rates...if I could have help
going thru the existing reflectors to determine their rates. Since the
rates aren't usually stated on the MOTD, that'd involve connecting to the
ref multiple times..changing your rates each time until you get an error
message from the ref..ie: not an easy task..and to do it for each of the 4
rates..would also tick off anyone on the ref seeing you come in and leave
about 30-40 times

--
streak@ccwf.cc.utexas.edu    * Jason Williams -- Austin, Tx.  |     |
streak@mail.utexas.edu       * University of Texas at Austin  | ___ |
streak@cs.utexas.edu         * BS Computer Science             \_|_/
*************** http://ccwf.cc.utexas.edu/~streak/ **************|