This would be interesting, I still don't know DNK database and codebase enough to
give opinions on triviality of this feature however.

How about breaking Top Users down into:

* Top Kicker (most kicks per timer period)
* Top Submitters (number of submissions per time period, might need to break it down
from published versus non-published)
* Top Commenters (maybe only give 1 pt per story instead of 1 pt per comment, so just
because someone posts 20 comments about 1 story they only get 1 point)

This might take a bit more of coding:

* Top domains/blogs
(would need to break down the submitted URLs to host domains and then compute totals.
So all entries from http://blog.incremental.ie would be grouped together and totaled.)

I was thinking a bit more about this.

How would something structured like this sound?

User could click on Reports (or whatever name) and see the total for today

Then they could click on a Year, Month, or other day of this month.

They could navigate to other year, then click on month (in case they wanted to jump
to Sept 2006, they click on 2006, then Sept).

On "/Reports/2007" page it shows
-Top 10 Kicked Stories
-Top 10 Kickers
-Top 10 Commenters
-Top 10 Submitters
-Top 10 Tags

Then a user could click on Sept and go to "/Reports/2007/09"
-Top 10 Kicked Stories
-Top 10 Kickers
-Top 10 Commenters
-Top 10 Submitters
-Top 10 Tags

Then a user could click on 10 and go to "/Reports/2007/09/10"
-Top 10 Kicked Stories
-Top 10 Kickers
-Top 10 Commenters
-Top 10 Submitters
-Top 10 Tags

I was thinking that the navigation would work similar to:
http://botd.wordpress.com/archives/

The URL rewriting would also work similar to the Blog of the Day Wordpress archives

Sounds great. 

I'll start this. The weekly reports will need to be added later along with the Google
Analytics piece.

But I'm stuck on a name for it. Should it be called:

* Top 10 Lists
* Kicks of the Year/Month/Day/Week
* Reports
* ?

Also, I don't know if we can find out how many users logged in during a time frame.
Doesn't the database just story last logon and creation times?

Maybe we can leave out the 'how many user logged on' count. We can still show how
many new users there were.

Do we need a name for this? DotNetKicks Zeitgeist works for me.

Ok. I had no idea what a Zeitgeist was. I had to look it up and found out that it's a
ghost of some sorts :) Google Zeitgeist was replaced a few months ago with Google Hot
Trends.

Should this data be written to a secondary table to improve performance? I wouldn't
want to run some of these statistics/computations every day or whenever the cache
timed out, because the values won't ever change (except for todays/this month/etc
which we can automate and just execute the script once a day to update all of the
values). But if someone is looking at June 2007 stats, we shouldn't need to recompute
those stats because they won't change.

I can come up with a table scheme and some procedures that will populate it. Then we
can just query the secondary/statistics table and cache those results.

A custom table for this data would be good. We are already running a background tasks
so we can hook in there. Thanks.

I've started work on this. should have something to commit later today. it won't be
ready for public consumption, but it will have the basics there. Once you add the new
SPs, can you generate the updated DAL for it? I'm just using a reader now and I'd
rather get a class created for it instead of relying on the reader["columnname"]
coding. I don't think I'll need a separate table for it.

Added top 10 kicked stories and top 10 commented on stories for Year and Year/Month.
Still need to add in caching before public consumption, but should work fine for
admins/moderators to test live.

Just thought of another category to add to the list:

Top 10 Quality Submitters (or something named to that affect). This is the top 10
users  who submitted stories received the most kicks per story submitted. 

Wanted to average it out, because one user could submit 30 stories for 30 points
versus  another user who submitted 2 stories with 10 kicks each. 

The second user (average 10 kicks per story) should be ranked above the first
(average 1 kick per story). 

Kicks would be counted per time period just like the others, so stories submitted at
the end of the month usually fair less well than earlier submitted stories.

I've just deployed the latest SVN revision to DNK - this Zeitgeist feature is
fantastic even at this early stage!

Screen shot attached for review, if users are looking here :)

I want to add Browse By Day to the Month page (so the browse by month bulleted list
will become every day of the month).

I'm also going to change the format of the bullets to a multi-column approach, so it
doesn't take up so much room in the bullet lists. Just working on back-end first then
will spend some time on the front-end.

	zeitgeist.PNG
	99 KB Download

Looks good. 

At some point we should create a new StoryListControl for displaying story titles and
a  small kick it widget.

It would be nice if there was a forward and backward navigation.

Yeah, I was thinking of making something like that, but didn't right away because
some of the top 10s won't be stories (could be users, etc.).

Maybe we could make something titled MiniStoryListControl? That does just the Title
with a popup bubble with the story description onhover. We could add a little Kick
box, similar to the other list control. We will also have different data to bind to
the control, so we can't use the same DAL class as the StoryList.

I agree on the navigation. I'm hoping to add that when I get to the front end. Wanted
to have a way of going left-to-right (month, year, day), or Up (day-to-month,
month-to-year), or down (year-to-month, month-to-day).

Here's a question to everyone. I was thinking about this a little more after Gavin
brought up a good point about the StoryList control.

Right now, the code does stats based on the time period. So if Story A was submitted
August 25 and got 5 kicks in Aug and 5 kicks in Sept, then it goes 5 for Aug and 5
for Sept. It counts the number of kicks during the time period (example here was
month). This will show the story with 5 kicks (in Aug and Sept), but it really has 10
kicks. So if we showed the story in mini-mode and show it with 10, it wouldn't make
sense because it might not be in order (since there could be stories with 6, 7, 8, 9
ranked above it).

Should we change this to just do totals based on when it was submitted versus when
each kick occurred? Same thing with comments, etc. Should it be based on story submit
date or each event (kick/comment) creation dates?

Perhaps we could display the total kick in the 'kick it' widget. We could the display
the number of kicks that occurred in the time period in italics beside the title.

Yeah, I thought about that too, but what about changing how the stats are computed so
that near period-end stories are given all their kicks. 

If we switch to using submitted date for the basis of stats instead of the individual
(kick/comment) date, then we don't the SPs since we can just use aggregate SubSonic
Queries. Then we can use the StoryList control (maybe remove a few items to make a
mini-StoryList control) and all of the items will show up in order of kicks.

I think my view point is switching away from my original ideal and towards the
submitted date. I see good stories submitted at the end of the month/year not getting
all of their recognition. But then it doesn't reflect accurately what was happening
during that time period because you can kick a story today that was submitted in July
and it might get raised on the list - whereas the original method would count the
tick towards today's month/year.

Good points. Lets switch to submitted date so.

I added a navigation system, but I'm not completely happy with it. It works, but I'm
having a hard time thinking of a better system. If someone has a good idea of a
better nav system, then please let me know or go ahead and make the change.

I removed the references to the stored procedures and I'm just using SubSonic.Query
now for both of the two list types. Still need to add a few more things (top tags,
users, etc.)

I think its functional now and it's using a new zeitgeist cache. I'm going to spend
the next few days on the CSS stuff, so feel free for someone to finish this with
whatever is needed.

	example.png
	36.9 KB Download

Oh, and we still need a mini-StoryList control. Preferably one that allows numbering,
so we can say 1, 2, 3, etc. or works within a OL.

That's great, I'll publish it to the site in a few hours.

updated this with some aggregate stats such as number of stories submitted,
published, comments, etc.

also updated the storylist and storysummary controls for a "mini" version.
ShowFullSummary=false (true is default and what is used on main pages). If false,
then it shows a mini-description. Such as the attached screenshot.

I don't know if I like this more than the basic list or not. Having a mini version
still takes up a lot of page space. Maybe we need to trim this down even further?

	mini_storylist.PNG
	45.8 KB Download

Perhaps we could implement a basic list similar to digg? Just the number of kicks and
the title.

r460

Added "Most Used Tags"

The query isn't pretty because I needed to use the SubSonic.QueryCommand due to lack
of JOIN and GROUP BY support in this release of SubSonic (hopefully it'll added soon
and we can update this simplify this code)

	mostusedtags.PNG
	9.6 KB Download

Added  "Number of User Registrations" and "Most Popular Domains"

Most Popular domains is the host domain (uri authority) of the submitted story ranked
by total number of kicks per domain.

Example: 
Most Popular Domains for September 2007
1. news.bbc.co.uk (90)
2. www.domain.com (14)
3. rte.ie (10)
4. www.europeantour.com (4)
5. dotnetkicks.com (1)

It would be great if Gavin (or anyone with a larger dataset) could run this on a
large dataset. I'm not sure of the sorting algorithm since I'm basically sorting a
keyvaluepair by the value instead of the key.

Do we need to add a browse by week? If so, would it be weeks per year (52) or weeks
per month (4 or 5), what about hanging weeks where month 1 ends on Wednesday would
that month week just be a couple of days or would it be more of a week based on year
(so it would always be 7 days except for first/last weeks).

Also, out of the above ideas that everyone has contributed, are there any that should
be added next? I guess we could keep adding more and more lists. I've been taking it
slow adding one or two things at a time - I don't like adding a bunch of stuff at
once since that increases the chances for bugs to slip through.

 Items mentioned above
 
 done
 * Top kicked stories 
 * Top tags
 * User stats (created an account)
 * Google analytics for the period
a weekly report from DNK.com would be quite useful - outside of Zeitgeist
* Top domains/blogs
  
cannot be done
* User stats (how many logged on) - need to log logons? can we do this with the new
user action table?
  
Not done
* Top users (dependent on some sort of scoring system)
* Top User Kicker 
* Top User Submitters 
* Top User Commenters 

The Zeitgest is progressing nicely. We'll have to make it a first class citizen in
the UI soon.

I'd be happy without weeks.

Perhaps we should create some graphs based on some of the data. When viewing a year
we could show a graph displaying the progress over the 12 months. When viewing a
month the graph would show the progress over the days in the month.

I have been thinking about that as well. 

The only downside is that we'll need to do more coding since we can't use the current
methods (current methods only return stats for top X entries and if LINQ is #1 in
Jan, it might be #11 in February and not added to the yearly totals, etc.)

I really wished there was a way to do Group By statements in Subsonic. Can you put in
a request with your friends there :) If there was a way, it would simplify the coding
process tremendously even if it only returned a IDataReader it would help. I can
provide code and examples of what we need it to do.

Do you have a preference in charting libraries? Or should we just roll our own? I've
made a couple .NET based graphing controls for work, but they were pretty simple
(probably as simple as we need). I was thinking of just using CSS for the bar charts
since that would just be client rendering and no additional server side overhead.

 

Added IsValidated to user queries for registration counts.
Added most published users and most published domains lists.
r495

examples:

 Most Published Users for September 2007

   1. user10 (6 stories)
   2. admin (4 stories)
   3. user3 (4 stories)
   4. user2 (3 stories)
   5. user1 (1 story)
   6. user4 (1 story)

 Most Published Domains for September 2007

   1. news.bbc.co.uk (14 stories)
   2. rte.ie (2 stories)
   3. www.domain.com (2 stories)
   4. www.europeantour.com (1 story)

Could you sort the years/months descending ? The latest month feels more natural to
be at top - at least to me :).

Maybe the Zeitgeist content should be displayed in the middle of the page instead of
under the left Calendar-like navigation? I'm using FF3, wondering why no one else
noticed/commented on it yet.