[MUSIC]
This lecture is about
a specific technique for
Contextual Text Mining called Contextual
Probabilistic Latent Semantic Analysis.
In this lecture, we're going to continue
discussing Contextual Text Mining.
And we're going to introduce Contextual
Probablitistic Latent Semantic Analysis
as exchanging of POS for
doing contextual text mining.
Recall that in contextual text mining
we hope to analyze topics in text,
in consideration of the context so
that we can associate the topics with a
property of the context were interesting.
So in this approach, contextual
probabilistic latent semantic analysis,
or CPLSA, the main idea is to
express to the add interesting
context variables into a generating model.
Recall that before when we generate
the text we generally assume we'll start
wIth some topics, and
then assemble words from some topics.
But here, we're going to add context
variables, so that the coverage of topics,
and also the content of topics
would be tied in context.
Or in other words, we're going to let
the context Influence both coverage and
the content of a topic.
The consequences that this will enable
us to discover contextualized topics.
Make the topics more interesting,
more meaningful.
Because we can then have topics
that can be interpreted as
specifically to a particular
context that we are interested in.
For example, a particular time period.
As an extension of PLSA model,
CPLSA does the following changes.
Firstly it would model the conditional
likelihood of text given context.
That clearly suggests that the generation
of text would then depend on context,
and that allows us to bring
context into the generative model.
Secondly, it makes two specific
assumptions about the dependency
of topics on context.
One is to assume that depending on
the context, depending on different time
periods or different locations, we assume
that there are different views of a topic
or different versions of word
descriptions that characterize a topic.
And this assumption allows
us to discover different
variations of the same topic
in different contexts.
The other is that we assume the topic
coverage also depends on the context.
That means depending on the time or
location, we might cover
topics differently.
Again, this dependency
would then allow us to
capture the association of
topics with specific contexts.
We can still use the EM algorithm to solve
the problem of parameter estimation.
And in this case, the estimated parameters
would naturally contain context variables.
And in particular,
a lot of conditional probabilities
of topics given certain context.
And this is what allows you
to do contextual text mining.
So this is the basic idea.
Now, we don't have time to
introduce this model in detail,
but there are references here that you
can look into to know more detail.
Here I just want to explain the high
level ideas in more detail.
Particularly I want to explain
the generation process.
Of text data that has context
associated in such a model.
So as you see here, we can assume
there are still multiple topics.
For example, some topics might represent
a themes like a government response,
donation Or the city of New Orleans.
Now this example is in the context
of Hurricane Katrina and
that hit New Orleans.
Now as you can see we
assume there are different
views associated with each of the topics.
And these are shown as View 1,
View 2, View 3.
Each view is a different
version of word distributions.
And these views are tied
to some context variables.
For example, tied to the location Texas,
or the time July 2005,
or the occupation of the author
being a sociologist.
Now, on the right side, now we assume
the document has context information.
So the time is known to be July 2005.
The location is Texas, etc.
And such context information is
what we hope to model as well.
So we're not going to just model the text.
And so one idea here is to model
the variations of top content and
various content.
And this gives us different views
of the water distributions.
Now on the bottom you will see the theme
coverage of top Coverage might also vary
according to these context
because in the case
of a location like Texas, people might
want to cover the red topics more.
That's New Orleans.
That's visualized here.
But in a certain time period,
maybe Particular topic and
will be covered more.
So this variation is
also considered in CPLSA.
So to generate the searcher document With
context, with first also choose a view.
And this view of course now could
be from any of these contexts.
Let's say, we have taken this
view that depends on the time.
In the middle.
So now, we will have a specific
version of word distributions.
Now, you can see some probabilities
of words for each topic.
Now, once we have chosen a view,
now the situation will be very similar
to what happened in standard ((PRSA))
We assume we have got word distribution
associated with each topic, right?
And then next, we will also choose
a coverage from the bottom, so
we're going to choose a particular
coverage, and that coverage,
before is fixed in PLSA, and
assigned to a particular document.
Each document has just one
coverage distribution.
Now here, because we consider context, so
the distribution of topics or the coverage
of Topics can vary depending on the
context that has influenced the coverage.
So, for example,
we might pick a particular coverage.
Let's say in this case we picked
a document specific coverage.
Now with the coverage and
these word distributions
we can generate a document in
exactly the same way as in PLSA.
So what it means, we're going to
use the coverage to choose a topic,
to choose one of these three topics.
Let's say we have picked the yellow topic.
Then we'll draw a word from this
particular topic on the top.
Okay, so
we might get a word like government.
And then next time we might
choose a different topic, and
we'll get donate, etc.
Until we generate all the words.
And this is basically
the same process as in PLSA.
So the main difference is
when we obtain the coverage.
And the word distribution,
we let the context influence our choice So
in other words we have extra switches
that are tied to these contacts that will
control the choices of different views
of topics and the choices of coverage.
And naturally the model we have
more parameters to estimate.
But once we can estimate those
parameters that involve the context,
then we will be able to understand
the context specific views of topics,
or context specific coverages of topics.
And this is precisely what we
want in contextual text mining.
So here are some simple results.
From using such a model.
Not necessary exactly the same model,
but similar models.
So on this slide you see
some sample results of
comparing news articles about Iraq War and
Afghanistan War.
Now we have about 30 articles on Iraq
wa,r and 26 articles on Afghanistan war.
And in this case,
the goal is to review the common topic.
It's covered in both sets of articles and
the differences of variations of
the topic in each of the two collections.
So in this case the context is explicitly
specified by the topic or collection.
And we see the results here
show that there is a common
theme that's corresponding to
Cluster 1 here in this column.
And there is a common theme indicting that
United Nations is involved in both Wars.
It's a common topic covered
in both sets of articles.
And that's indicated by the high
probability words shown here, united and
nations.
Now if you know the background,
of course this is not surprising and
this topic is indeed very
relevant to both wars.
If you look at the column further and
then what's interesting's that the next
two cells of word
distributions actually tell us
collection specific variations
of the topic of United Nations.
So it indicates that the Iraq War,
United Nations was more involved
in weapons factions, whereas in
the Afghanistan War it was more involved
in maybe aid to Northern Alliance.
It's a different variation of
the topic of United Nations.
So this shows that by
bringing the context.
In this case different the walls or
different the collection of texts.
We can have topical variations
tied to these contexts,
to review the differences of coverage
of the United Nations in the two wars.
Now similarly if you look at
the second cluster Class two,
it has to do with the killing of people,
and, again,
it's not surprising if you know
the background about wars.
All the wars involve killing of people,
but
imagine if you are not familiar
with the text collections.
We have a lot of text articles, and
such a technique can reveal the common
topics covered in both sets of articles.
It can be used to review common topics
in multiple sets of articles as well.
If you look at of course in
that column of cluster two,
you see variations of killing of people
and that corresponds to different contexts
And here is another example of results
obtained from blog articles
about Hurricane Katrina.
In this case,
what you see here is visualization of
the trends of topics over time.
And the top one shows just
the temporal trends of two topics.
One is oil price, and one is about
the flooding of the city of New Orleans.
Now these topics are obtained from
blog articles about Hurricane Katrina.
And people talk about these topics.
And end up teaching to some other topics.
But the visualisation shows
that with this technique,
we can have conditional
distribution of time.
Given a topic.
So this allows us to plot
this conditional probability
the curve is like what you're seeing here.
We see that, initially, the two
curves tracked each other very well.
But later we see the topic of New Orleans
was mentioned again but oil price was not.
And this turns out to be
the time period when another hurricane,
hurricane Rita hit the region.
And that apparently triggered more
discussion about the flooding of the city.
The bottom curve shows
the coverage of this topic
about flooding of the city by block
articles in different locations.
And it also shows some shift of
coverage that might be related to
people's migrating from the state
of Louisiana to Texas for example.
So in this case we can see the time can
be used as context to review trends of
topics.
These are some additional
results on spacial patterns.
In this case it was about
the topic of government response.
And there was some criticism about
the slow response of government
in the case of Hurricane Katrina.
And the discussion now is
covered in different locations.
And these visualizations show the coverage
in different weeks of the event.
And initially it's covered
mostly in the victim states,
in the South, but then gradually
spread into other locations.
But in week four,
which is shown on the bottom left,
we see a pattern that's very similar
to the first week on the top left.
And that's when again
Hurricane Rita hit in the region.
So such a technique would allow
us to use location as context
to examine their issues of topics.
And of course the moral
is completely general so
you can apply this to any
other connections of text.
To review spatial temporal patterns.
His view found another application
of this kind of model,
where we look at the use of the model for
event impact analysis.
So here we're looking at the research
articles information retrieval.
IR, particularly SIGIR papers.
And the topic we are focusing on
is about the retrieval models.
And you can see the top words with high
probability about this model on the left.
And then we hope to examine
the impact of two events.
One is a start of TREC, for
Text and Retrieval Conference.
This is a major evaluation
sponsored by U.S.
government, and was launched in 1992 or
around that time.
And that is known to have made a impact on
the topics of research
information retrieval.
The other is the publication of
a seminal paper, by Croft and Porte.
This is about a language model
approach to information retrieval.
It's also known to have made a high
impact on information retrieval research.
So we hope to use this kind of
model to understand impact.
The idea here is simply to
use the time as context.
And use these events to divide
the time periods into a period before.
For the event and
another after this event.
And then we can compare
the differences of the topics.
The and the variations, etc.
So in this case,
the results show before track the study of
retrieval models was mostly a vector
space model, Boolean model etc.
But the after Trec,
apparently the study of retrieval models
have involved a lot of other words.
That seems to suggest some
different retrieval tasks, so for
example, email was used in
the enterprise search tasks and
subtopical retrieval was another
task later introduced by Trec.
On the bottom,
we see the variations that are correlated
with the propagation of
the language model paper.
Before, we have those classic
probability risk model,
logic model, Boolean etc., but after 1998,
we see clear dominance of language
model as probabilistic models.
And we see words like language model,
estimation of parameters, etc.
So this technique here can use events as
context to understand the impact of event.
Again the technique is generals so
you can use this to analyze
the impact of any event.
Here are some suggested readings.
The first is paper about simple staging of
psi to label cross-collection comparison.
It's to perform comparative
text mining to allow us to
extract common topics shared
by multiple collections.
And there are variations
in each collection.
The second one is the main
paper about the CPLSA model.
Was a discussion of a lot of applications.
The third one has a lot of details
about the special temporal patterns for
the Hurricane Katrina example.
[MUSIC]

