Algoliaのドキュメントに、Algolia Answersの章が3月22日に追加されました！
Algolia Answersはインデックスのデータをセマンティックに検索することができる、というものですが、それはNLU(自然言語学習)という技術を元にして、ユーザーによるクエリと、インデックスのレコードを分析して、機械学習を使って、コンテンツの中から最もRelevantな回答をする、というものになります。
例えば、”healthy soup”というクエリでレシピブログを検索するとします。Answersは”Bone Broth Strengthens Immunity, Body, and Joints.”(骨だしは、免疫力、身体、そして、間接を強化する)といったブログを返すことができます。このブログのタイトルの中にはクエリで使われたキーワードは出てきませんが、セマンティックな検索をすることで、コンテンツ内の関連性を導き出して、答えを瞬時に見つけることができます。
※ 注: 2021年3月23日現在、こちらの機能はLimited Releaseとなっています。コチラからWaiting Listにご登録ください。
はじめましょう
認証
Answers APIへのアクセスができるようになったら、nluReadAnswers ACLのAPI keyを作成します。ここでは nluReadAnswers と search パーミッションを与えるようにしましょう。
データセット
記事や文字起こしなどの長文のテキストがAnswersには適しています。
インデックスの中から答えを見つける
クレデンシャルの設定をしたら、Answers APIにリクエストを送ることができます:
const data = {
"query": "when do babies start learning?",
"queryLanguages": ["en"],
"attributesForPrediction": ["title", "description", "transcript"],
"nbHits": 2
};
const URL = `https://${APPLICATION_ID}-dsn.algolia.net/1/answers/${INDEX_NAME}/prediction`;
fetch(URL, {
method: "POST",
headers: {
"X-Algolia-Application-Id": ${APPLICATION_ID},
"X-Algolia-API-Key": ${API_KEY_WITH_ANSWERS_ACL}
},
body: JSON.stringify(data)
})
.then((response) => response.json())
.then((res) => displayResults(res.hits))
.catch(console.error);レスポンスはAlgoliaの検索レスポンスと同様ですが、 _answer というフィールドがそれぞれのhitに追加されます。このフィールドは extract というクエリに回答するhitを保持し、その信頼度スコア(confidence score)と元の属性が extractAttribute としてhitの中に含まれます。
※ 注 もし、クエリに対してマッチする回答がなかった場合Answersは空の hits 配列を返します。
レスポンスの例
{
"nbPages": 1,
"nbHits": 2,
"params": "query=when+do+babies+start+learning%3F&naturalLanguages=%5B%22en%22%5D&hitsPerPage=150&analyticsTags=%5B%22algolia-answers%22%5D&queryType=prefixNone&attributesToSnippet=%5B%22description%3A80%22%2C%22title%3A80%22%2C%22transcript%3A80%22%5D&restrictSearchableAttributes=%5B%22description%22%2C%22title%22%2C%22transcript%22%5D&queryLanguages=%5B%22en%22%5D&ignorePlurals=en&removeStopWords=en&removeWordsIfNoResults=allOptional&ruleContexts=natural_language&analyticsTags=algolia-answers%2Cnatural_language",
"hitsPerPage": 150,
"query": "when do babies start learning?",
"exhaustiveNbHits": true,
"hits": [
{
"description": "Pop quiz: When does learning begin? Answer: Before we are born. Science writer Annie Murphy Paul talks through new research that shows how much we learn in the womb -- from the lilt of our native language to our soon-to-be-favorite foods.",
"duration": 1006,
"languages": 37,
"main_speaker": "Annie Murphy Paul",
"name": "Annie Murphy Paul: What we learn before we're born",
"speaker_occupation": "Science author",
"tags": [
"brain",
"children",
"culture",
"education",
"evolution",
"food",
"mental health",
"neuroscience",
"pregnancy",
"science",
"writing"
],
"title": "What we learn before we're born",
"url": "https://www.ted.com/talks/annie_murphy_paul_what_we_learn_before_we_re_born\n",
"transcript": "French babies cry on a rising note while German babies end on a falling note, imitating the melodic contours of those languages. Now why would this kind of fetal learning be useful? It may have evolved to aid the baby's survival. From the moment of birth, the baby responds most to the voice of the person who is most likely to care for it — its mother. It even makes its cries sound like the mother's language, which may further endear the baby to the mother, and which may give the baby a head start in the critical task of learning how to understand and speak its native language.But it's not just sounds that fetuses are learning about in utero. It's also tastes and smells. By seven months of gestation, the fetus' taste buds are fully developed, and its olfactory receptors, which allow it to smell, are functioning. The flavors of the food a pregnant woman eats find their way into the amniotic fluid, which is continuously swallowed by the fetus. Babies seem to remember and prefer these tastes once they're out in the world. In one experiment, a group of pregnant women was asked to drink a lot of carrot juice during their third trimester of pregnancy, while another group of pregnant women drank only water. Six months later, the women's infants were offered cereal mixed with carrot juice, and their facial expressions were observed while they ate it. The offspring of the carrot juice drinking women ate more carrot-flavored cereal, and from the looks of it, they seemed to enjoy it more.A sort of French version of this experiment was carried out in Dijon, France where researchers found that mothers who consumed food and drink flavored with licorice-flavored anise during pregnancy showed a preference for anise on their first day of life, and again, when they were tested later, on their fourth day of life. Babies whose mothers did not eat anise during pregnancy showed a reaction that translated roughly as \"yuck.\" What this means is that fetuses are effectively being taught by their mothers about what is safe and good to eat. Fetuses are also being taught about the particular culture that they'll be joining through one of culture's most powerful expressions, which is food. They're being introduced to the characteristic flavors and spices of their culture's cuisine even before birth.Now it turns out that fetuses are learning even bigger lessons. But before I get to that, I want to address something that you may be wondering about. The notion of fetal learning may conjure up for you attempts to enrich the fetus — like playing Mozart through headphones placed on a pregnant belly. But actually, the nine-month-long process of molding and shaping that goes on in the womb is a lot more visceral and consequential than that. Much of what a pregnant woman encounters in her daily life — the air she breathes, the food and drink she consumes, the chemicals she's exposed to, even the emotions she feels — are shared in some fashion with her fetus. They make up a mix of influences as individual and idiosyncratic as the woman herself. The fetus incorporates these offerings into its own body, makes them part of its flesh and blood. And often it does something more. It treats these maternal contributions as information, as what I like to call biological postcards from the world outside.So what a fetus is learning about in utero is not Mozart's \"Magic Flute\" but answers to questions much more critical to its survival. Will it be born into a world of abundance or scarcity? Will it be safe and protected, or will it face constant dangers and threats? Will it live a long, fruitful life or a short, harried one? The pregnant woman's diet and stress level in particular provide important clues to prevailing conditions like a finger lifted to the wind. The resulting tuning and tweaking of a fetus' brain and other organs are part of what give us humans our enormous flexibility, our ability to thrive in a huge variety of environments, from the country to the city, from the tundra to the desert.",
"objectID": "299355000",
"_answer": {
"extract": "Pop quiz: When does <em>learning</em> begin? Answer: Before we are born. Science writer Annie Murphy Paul talks through new research that shows how much we learn in the womb -- from the lilt of our native language to our soon-to-be-favorite foods.",
"score": 212.239,
"extractAttribute": "description"
}
},
{
"description": "Pawan Sinha details his groundbreaking research into how the brain's visual system develops. Sinha and his team provide free vision-restoring treatment to children born blind, and then study how their brains learn to interpret visual data. The work offers insights into neuroscience, engineering and even autism.",
"duration": 1103,
"languages": 30,
"main_speaker": "Pawan Sinha",
"name": "Pawan Sinha: How brains learn to see",
"speaker_occupation": "Visual neuroscientist",
"tags": [
"biology",
"health care",
"medicine",
"neuroscience",
"science",
"sight"
],
"title": "How brains learn to see",
"url": "https://www.ted.com/talks/pawan_sinha_on_how_brains_learn_to_see\n",
"transcript": "If you are a blind child in India, you will very likely have to contend with at least two big pieces of bad news. The first bad news is that the chances of getting treatment are extremely slim to none, and that's because most of the blindness alleviation programs in the country are focused on adults, and there are very, very few hospitals that are actually equipped to treat children. In fact, if you were to be treated, you might well end up being treated by a person who has no medical credentials as this case from Rajasthan illustrates. This is a three-year-old orphan girl who had cataracts. So, her caretakers took her to the village medicine man, and instead of suggesting to the caretakers that the girl be taken to a hospital, the person decided to burn her abdomen with red-hot iron bars to drive out the demons. The second piece of bad news will be delivered to you by neuroscientists, who will tell you that if you are older than four or five years of age, that even if you have your eye corrected, the chances of your brain learning how to see are very, very slim — again, slim or none.So when I heard these two things, it troubled me deeply, both because of personal reasons and scientific reasons. So let me first start with the personal reason. It'll sound corny, but it's sincere. That's my son, Darius. As a new father, I have a qualitatively different sense of just how delicate babies are, what our obligations are towards them and how much love we can feel towards a child. I would move heaven and earth in order to get treatment for Darius, and for me to be told that there might be other Dariuses who are not getting treatment, that's just viscerally wrong. So that's the personal reason.Scientific reason is that this notion from neuroscience of critical periods — that if the brain is older than four or five years of age, it loses its ability to learn — that doesn't sit well with me, because I don't think that idea has been tested adequately. The birth of the idea is from David Hubel and Torsten Wiesel's work, two researchers who were at Harvard, and they got the Nobel Prize in 1981 for their studies of visual physiology, which are remarkably beautiful studies, but I believe some of their work has been extrapolated into the human domain prematurely. So, they did their work with kittens, with different kinds of deprivation regiments, and those studies, which date back to the '60s, are now being applied to human children.So I felt that I needed to do two things. One: provide care to children who are currently being deprived of treatment. That's the humanitarian mission. And the scientific mission would be to test the limits of visual plasticity. And these two missions, as you can tell, thread together perfectly. One adds to the other; in fact, one would be impossible without the other. So, to implement these twin missions, a few years ago, I launched Project Prakash. Prakash, as many of you know, is the Sanskrit word for light, and the idea is that in bringing light into the lives of children, we also have a chance of shedding light on some of the deepest mysteries of neuroscience. And the logo — even though it looks extremely Irish, it's actually derived from the Indian symbol of Diya, an earthen lamp. The Prakash, the overall effort has three components: outreach, to identify children in need of care; medical treatment; and in subsequent study. And I want to show you a short video clip that illustrates the first two components of this work.This is an outreach station conducted at a school for the blind.(Text: Most of the children are profoundly and permanently blind ...)Pawan Sinha: So, because this is a school for the blind, many children have permanent conditions. That's a case of microphthalmos, which is malformed eyes, and that's a permanent condition; it cannot be treated. That's an extreme of micropthalmos called enophthalmos. But, every so often, we come across children who show some residual vision, and that is a very good sign that the condition might actually be treatable.",
"objectID": "301167000",
},
"_answer": {
"extract": "the chances of your brain <em>learning</em> how to see are very, very slim — again, slim or none.So when I heard these two things, it troubled me deeply, both because of personal reasons and scientific reasons. So let me first <em>start</em> with the personal reason. It'll sound corny, but it's sincere. That's my son, Darius. As a new father, I have a qualitatively different sense of just how delicate <em>babies</em> are, what our obligations are towards them",
"score": 152.885,
"extractAttribute": "transcript"
}
}
],
"processingTimeMS": 56,
"page": 0
}こちらの例では、簡潔にするために _highlightResult と _snippetResult をレスポンスの中から取り除いています。
※ 注 このレスポンスには、同じレコードから2つの回答(Answers)があります: Answersは、1つのレコードのコンテンツのいくつかの部分がクエリに適している場合において、複数の回答を返すことが可能です。
複数の回答を取得
nbHits パラメータを使って、1つのクエリに対する複数の回答を取得することができます。例えば、上述のスニペットであれば、nbHitsパラメータが2ということで、2つの回答を取得しています。複数の回答を取得した場合、Answersは信頼度スコアが大きい方から小さい方の順番(desc)でソートを行います。
threshold を使った信頼度スコアによる結果のフィルタリング
threshold パラメータを使って信頼度スコアによる結果のfine-tuneを行うことができます。Answersは、_answerフィールドを持つ結果のうち、threshold(しきい値)より低いスコアのものを取り除くことができます。
少なくとも1つの回答を得るためには、thresholdを 0 に設定すべきです。クエリと関連がない結果に関してのみ、信頼度スコアはマイナスになります。thresholdを185に設定すると、Answersエンジンが関連性が高いと予測をしたhitsのみを取得することができます。
※ 注 score はクエリに依存します。同じクエリに対する様々な回答のスコアを比較することは可能ですが、異なるクエリに対する回答のスコアを比較することは意味があるとは言えません。
ハイライトのコントロール
デフォルトでは、_highlightResult で一致する単語にハイライトがあてられますが、クエリ実行時に attributesToHighlight を searchParameters の中で設定することにより highlighting をコントロールすることができます。
検索Analyticsの管理
ユーザーがどのように検索を行ったかを理解し、セマンティック検索のインパクトを測定するために、Algolia Answersは Search Analytics とインテグレートしています。
デフォルトではAnswers APIにリクエストを送る際に analyticsTags: "algolia-answers"がAlgoliaによってタグ付けされます。analyticsTagsには searchParameters パラメータの中でタグを追加していただくこともできます。
コメント